Removing some lines from a big text file ?Closed

Question

Hello guys, i have this BIG text file (50 000 lines), each line contains a name, example:

Michael johnson
derrek pierre
joséph francois
saphire moore

i need to remove all the lines which contain international charcters like "é" , "à" .. and keep online lines with normal english alphabet, can you please suggest me the simplest way to do this ?

thanks a lot of your help.

Richard.Williams · Accepted Answer

This script will remove all lines that contain any international char. Basically, it ensures that each line has characters only from a-z, A-Z, 0-9, space, tab, comma, colon, semicolon.




# Script IntlNames.txt
str file, content, line, approvedlist
# Approved list of characters. Enclose this list between ^(#...)^.
set $approvedlist = "^(#a-zA>Z0>9 \,\:\;)^"
# Get file contents into a variable.
cat $file > $content
# Go thru lines one by one.
lex "1" $content > $line
while ($line <> "")
do
    # Does this line contain any character outside our "approved" list of characters ?
    if ( { chen -r $approvedlist $line } == 0)
        # No unapproved chars - this is a "good" line.
        echo $line
    endif
    # Get the next line
    lex "1" $content > $line
done




Save the script in file C:/Scripts/IntlNames.txt, start biterscripting, enter this command.


script "C:/Scripts/IntlNames.txt" file("C:/testfile.txt") > "C:/testfile.txt"


This will remove all lines with intl chars from file C:/testfile.txt. 50000 lines is pretty small file.

Removing some lines from a big text file ?

1 response

Similar discussions

Subscribe To Our Newsletter!