How to fix unicode REPLACEMENT CHARACTER � in postgres database

The question:

Is there a way fix the � problem in postgres database:

How to fix unicode REPLACEMENT CHARACTER � in postgres database

As you can see there are names with accents and are not displayed corretly in the database.
I would like to know if there is a way to fix this without dropping the database and restoring it… ?

When I execute

SELECT CAST (userfirstname AS bytea) FROM tab WHERE id = 42;`

this is the result i get:

               bytea
--------------------------------------
 x5374efbfbd7068616e696520284d6d6529
(1 ligne)

The Solutions:

Below are the methods you can try. The first solution is probably the best. Try others if the first one doesn’t work. Senior developers aren’t just copying/pasting – they read the methods carefully & apply them wisely to each case.

Method 1

That character (0xEFBFBD in UTF-8) really is Unicode 0xFFFD, the “replacement character” �.

So if that’s not what you want, the problem must have occurred when you inserted the data into the database. PostgreSQL does not surreptitiously modify strings, so some software most have done that before the data were put into the database.

Since the characters you would like to see are not all equal (I spot é and ç, for example), there is no automatic way to repair the damage.


All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Comment