Subject Re: CSV line termination issue
From Mervyn Bick <invalid@invalid.invalid>
Date Fri, 9 Oct 2020 09:37:49 +0200
Newsgroups dbase.getting-started

On 2020/10/09 00:21, Gaetano wrote:
>
> Hi All,
>
> I have an input CSV file that has a CHR(10) (\n - hard line feed) at the
> end of each line. When I f.gets(1000), dbase returns the first 1000
> characters and ignores that end of line character.
>
> When I f.gets(1000,chr(10)), it works, but then it may not handle
> correctly a CSV file created in Windows.
>
> What is a good way to handle an unknown end of line character? How do I
> test for the end of line character since gets() stops at the first end
> of line character it finds? I tried the following
>
..........
> The input sources should be limited to Windows, Mac, Android, iOS and
> Linux.

Personally, I prefer dedicated input programs for "special" .csv files
rather than trying to change the .csv file to a more standard layout.
The following code does, however, serve as a base for either approach.

If you decide on a dedicated input program  breakstring() from
stringex.cc in the dUFLP will take care of all the heavy lifting. :-)


***************************************
cInfile = 'whatever.csv'
f = new file()
f.open(cInfile)
cRead = f.read(10000)
eol = get_eol_marker(cRead)
f.seek(0)
do while not f.eof()
    cRead = f.gets(10000,eol)
//   ?cRead
    //Either add code here to parse and insert data into table
    //or use fNew.puts() to write cRead to a new file.
    //fNew.puts(cRead) will automatically use CRLF as the end of line marker
enddo
f.close()


function get_eol_marker(cStr)
    local eol_marker
    if  at(chr(13)+chr(10),cStr) > 0
       eol_marker = chr(13)+chr(10)  //Windows
    elseif at(chr(13),cStr) > 0
       eol_marker = chr(13)     //Old Mac Os
    elseif at(chr(10),cStr) > 0
       eol_marker = chr(10)  // Unix, Android or new Mac OS
    elseif at(chr(141),cStr) > 0
       eol_marker = chr(141)
    elseif at(chr(255),cStr) > 0
       eol_marker = chr(255)
    elseif at(chr(138),cStr) > 0
       eol_marker = chr(138)
    elseif at(chr(0),cStr) > 0
       eol_marker = chr(0)
    else
       msgbox('Unknown end of line marker','Error')
    endif
    return eol_marker
**********************************

If you ever come across a different eol marker simply add it to the
function.

Mervyn.