[3.4] bpo-32072: Fix issues with binary plists. (GH-4455)

[serhiy-storchaka]

• Fixed saving bytearrays.
• Identical objects will be saved only once.
• Equal references will be load as identical objects.
• Added support for saving and loading recursive data structures..
  (cherry picked from commit a897aee)

https://bugs.python.org/issue32072

[larryhastings]

I'm not qualified to review this change. Can you find another core dev who is?

[serhiy-storchaka]

@ronaldoussoren is a plistlib expert. His approved changes in 3.7 (#4455). Backports to 3.4 and 3.5 are straightforward.

[ronaldoussoren]

This appears to be a backport to python 3.4. AFAIK that version is in security-fix-only mode, and this is not a security fix.

Other than that I'm fine with backporting.

[serhiy-storchaka]

I think this is a security issue. The bug allows to organize a DDOS attack if the Python application reads Plist files from untrusted sources. For example LLVM Performance Tracking Software can be vulnerable.

[ronaldoussoren]

I didn't expect that anyone would use plist files like this, they are a crummy interchange format. Sigh...

I'm not entirely convinced about the DDOS potential, the recursion should be stopped before the interpreter crashes and anyone reading nested data structures should already deal with unwanted recursion: I could just as easily send a plist file that contains a deeply nested list. An uncompressed XML plist file containing a array nested 1200 deep is still only about 18K bytes long, and should easily nest deep enough to hit the recursion limit.

[serhiy-storchaka]

The problem is not only with the deep recursion, but with consuming a large amount of memory. For XML plist file and for other uncompressed data formats the consumed memory is linear from the size of the file. For binary plist file it can be exponential (O(2^N)). A 200-byte long plist file is enough for producing multiple gigabytes or terrabytes structures in memory. This is different from other common data formats.

[serhiy-storchaka]

For example b'bplist00\xa2\x01\x01\xa2\x02\x02\xa2\x03\x03\xa2\x04\x04\xa2\x05\x05\xa2\x06\x06\xa2\x07\x07\xa2\x08\x08\xa2\t\t\xa2\n\n\xa2\x0b\x0b\xa2\x0c\x0c\xa2\r\r\xa2\x0e\x0e\xa2\x0f\x0f\xa2\x10\x10\xa2\x11\x11\xa2\x12\x12\xa2\x13\x13\xa2\x14\x14\xa2\x15\x15\xa2\x16\x16\xa0\x08\x0b\x0e\x11\x14\x17\x1a\x1d #&),/258;>ADGJ\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x17\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00K' (130 bytes) is expanded to almost 1 GB in Python 3.4 and 3.5.

[ronaldoussoren]

That is a good reason to see this as a security issue, memory attacks are much harder to defend against.

I'm OK with applying this patch to 3.4.

[serhiy-storchaka]

@larryhastings ?