Lingua-e
Python

What is a UnicodeDecodeError?

← Errors

A UnicodeDecodeError is raised when Python tries to decode a sequence of bytes into a string using a given encoding (such as UTF-8) but encounters bytes that are not valid for that encoding. This is common when reading files saved with a different character set.

Why it happens

  • •Opening a file that was saved with Latin-1 or Windows-1252 encoding while Python defaults to UTF-8.
  • •Reading a binary file (image, PDF, executable) as if it were plain text.
  • •A file that mixes encodings, for example an otherwise UTF-8 file with a few legacy-encoded characters.
  • •Network data or database content containing raw bytes outside the ASCII range.

Examples

Python

Code that triggers the error

with open("report.csv") as f:
    content = f.read()

Error output

Traceback (most recent call last):
  File "app.py", line 2, in <module>
    content = f.read()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 42: invalid continuation byte

How to fix it

Always specify the encoding explicitly when opening files: open('file.txt', encoding='utf-8'). If the encoding is unknown, use the chardet library to detect it. Pass errors='replace' or errors='ignore' as a last resort when data loss is acceptable.

Fixed code

# Specify the correct encoding, or use errors='replace' as a fallback
with open("report.csv", encoding="latin-1") as f:
    content = f.read()

# If encoding is unknown, detect it first:
# pip install chardet
import chardet
raw = open("report.csv", "rb").read()
detected = chardet.detect(raw)
print(detected["encoding"])  # e.g. 'ISO-8859-1'

Related errors

Ready to practice your English at work?

Lingua-e has interactive exercises built around real developer conversations: standups, code reviews, retrospectives, and more. Practice until it comes naturally.

Try Lingua-e for free