Last active
July 3, 2025 03:29
-
-
Save szydan/b225749445b3602083ed to your computer and use it in GitHub Desktop.
<U+FEFF> character showing up in files. How to remove them?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 1) In your terminal, open the file using vim: | |
| vim file_name | |
| 2) Remove all BOM characters: | |
| :set nobomb | |
| 3) Save the file: | |
| :wq | |
| // taken from: | |
| http://stackoverflow.com/questions/7297888/ufeff-character-showing-up-in-files-how-to-remove-them | |
| Another recipe using awk | |
| http://stackoverflow.com/questions/1068650/using-awk-to-remove-the-byte-order-mark | |
| awk '{ if (NR==1) sub(/^\xef\xbb\xbf/,""); print }' INFILE > OUTFILE | |
See this StackOverflow Answer for a better place for this information: https://stackoverflow.com/a/15137601/293280
Worked perfectly, thank you!
Thank you!
thanks @Sebastian-D
thank you for this.
A quick method if you don;t want to go into code. Just copy the file content and paste it in gedit (or notepad) editor. Copy the same content again from editor and replace in original file. You will get the clean file.
Thanks a lot!
you are a Fucking legend!
awesome, many thanks for this gist! @Sebastian-D also thanks to you. i used your awk command to solve my problem :)
thamks
In a script:
#!/bin/bash
awk '{ gsub(/\xef\xbb\xbf/,""); print }' $1 > /tmp/feff && mv /tmp/feff $1
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This helped me.
The vim method did not work. The awk recipe only removes <U+FEFF> (or as it is shown in vim) from the first column. I modified it to remove ALL <U+FEFF>
awk '{ gsub(/\xef\xbb\xbf/,""); print }' INFILE > OUTFILE