What is a UnicodeDecodeError?

A UnicodeDecodeError is raised when Python tries to decode a sequence of bytes into a string using a given encoding (such as UTF-8) but encounters bytes that are not valid for that encoding. This is common when reading files saved with a different character set.

Why it happens

• •Opening a file that was saved with Latin-1 or Windows-1252 encoding while Python defaults to UTF-8.
• •Reading a binary file (image, PDF, executable) as if it were plain text.
• •A file that mixes encodings, for example an otherwise UTF-8 file with a few legacy-encoded characters.
• •Network data or database content containing raw bytes outside the ASCII range.

Examples

with open("report.csv") as f:
    content = f.read()

Traceback (most recent call last):
  File "app.py", line 2, in <module>
    content = f.read()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 42: invalid continuation byte

# Specify the correct encoding, or use errors='replace' as a fallback
with open("report.csv", encoding="latin-1") as f:
    content = f.read()

# If encoding is unknown, detect it first:
# pip install chardet
import chardet
raw = open("report.csv", "rb").read()
detected = chardet.detect(raw)
print(detected["encoding"])  # e.g. 'ISO-8859-1'

Related errors

• ValueError
• TypeError