Exiv2 returns "binary data" for valid content in Exif.Photo.UserComment

[ghost]

In Exiv2.0.27.3, source file value.cpp, function CommentValue::comment was changed. The modification targets to identify invalid binary data in a comment field with “Undefined” character code.
Yet, the implementation creates new problems. So, now just content encoded 7-bit ASCII in the character range 32-127 will be allowed.
Most (if not all) of the regional Windows or other ASCII codepages allow for special characters for their related cultures, represented in the character range 128-255. The Exif Specifications JEITA CP-3451 Exif2-2.PDF Table 7 do not restrict content to 7-Bit ASCII. On the contrary, for “Undefined” character code, it allows for regional settings, says “Depends on the localized PC in each country”, … “although the possibility of unreadable characters exist”.
Clearly, ASCII encoding in any codepage will not allow for intercultural exchange, if the encoding codepage is not known. But reducing allowed content to 7-Bit ASCII cannot be the solution.

There is another problem with the function: The subroutine “isBinary” which is used by CommentValue::comment causes problems, even if the “undefined” comment is 7-Bit ASCII encoded:
In case the encoding program (in this case Canon ZoomBrowser EX) writes a buffer to the field, where the rest of the buffer behind the string is properly filled with low-value, “isBinary” returns “true” as soon as it hits on the low-value in the loop over the string buffer with the consequence, the field is marked “binary comment” .

Suggestion: remove modifications in CommentValue::comment and, where “invalid input” might have caused problems, e.g. in print functions, as indicated in a comment referring to non-printable bytes, handle possible problems there (e.g. just replacing control characters or what else might be undesired in output formatting, but retaining the data). The program or routine that obtains the data from the library should deal with possible "non-printable" bytes when printing, if there are any ...

Thanks!

[clanmills]

Thanks for reporting this. Can you provide a file which has been modified by Canon ZoomBrowser EX in the way you describe, please?

You are not the first to have complained about this and it will get attention on the 0.27-maintenance branch. After I retire on 2021-01-18, the future of Exiv2 is out of my hands. At the moment, nobody has expressed any interest in becoming the maintainer. While this is a disappointment to me, the community is mistaken if they think I will continue to work on Exiv2. https://discuss.pixls.us/t/maintainer-urgently-needed-for-exiv2/21547/8

I am writing a book Image Metadata and Exiv2 Architecture https://clanmills.com/exiv2/book/

I leave Exiv2 with no open issues on the 0.27-maintenance branch. No security issue has been reported in 2020. The number of contributors is steadily increasing, the code is robust. We have excellent documentation and a solid release process. I am willing to support and mentor anyone who volunteers.

[clanmills]

Fix submitted: #1432. I have explain in the PR the change and reasoning. If you wish to discuss further, please do so in #1432.