Improve HEIC conversion

[tagatac]

Several possible routes here:

• use my own fork of goheif
• shell out to use magick
• compile libheif into bagoup
• reimplement relevant portions of libheif in bagoup

Three main benefits:

• improve stability
• get rid of constant-conversion warnings
• expose an interface for easy testing

magick appears to be stable at the cost of adding another system dependency.

[richardsprague]

My conversion stalled in the middle with this error:

decode HEIC image: error reading "ftyp" box: got box type "$\x14Ex" instead

[tagatac]

Ah, yes, that could also stand to be a warning (just referencing the HEIC file in the PDF when the conversion fails). I'll make that change.

@richardsprague, the full error message should give you the name of the file that bagoup is trying to convert. A couple of questions:

• Are you able to open that file, and does the image look okay?
• Does imagemagick conversion work?

brew install ghostscript imagemagick
magick <file.heic> <file.jpg>

• If it is not an overly sensitive file, do you mind sharing it ([email protected]) so that I can try it out in some solutions? I won't share it publicly. If you are not comfortable with this, the answers to the first two questions would still be very helpful.

[richardsprague]

1. No, I cannot open the (PDF) file that bagoup created. Yes, I can open the HEIC file.
2. Yes, magick conversion works correctly and I'm able to open and view the resulting JPG.

ERROR: export chats: chat file "messages-export/+1206xxxxxxx/iMessage;-;+1206xxxxxxx;;;SMS;-;+1206xxxxxxx.pdf" - message 16871: convert HEIC file "~/Library/Messages/Attachments/be/14/at_0_616F6C1C-C506-4589-95D2-90E0D81D186B/IMG_0321.HEIC" to JPG: decode HEIC image: error reading "ftyp" box: got box type "$\x14Ex" instead

[tagatac]

Thanks for that info. That's great that magick conversion works. It seems like this is probably the way to go for the issue in the OP.

Just to confirm, both the HEIF and JPG files are openable and look good in Preview?

[richardsprague]

Yes, both HEIC and the JPG converted by Magick are openable and look fine.

[tagatac]

Thanks! I think this anecdote gives enough motivation to abandon goheif when combined with its other issues

• lack of a supported fork
• superfluous warnings on compilation: Compiler warnings (linux) jdeng/goheif#14
• inability to vendor properly: cmd/go: go mod -vendor prunes non-package directories golang/go#26366
• copied code from liibde265 instead of using submodules: Use git submodules for external code jdeng/goheif#21

[tagatac]

Hmm, magick seems to add nearly 30 seconds to the run time when converting 57 images on my MacBook 😕

BAGOUP RESULTS:
Export folder: "testgoheif"
Export files written: 45
Chats exported: 48
Messages exported: 2154
Attachments referenced or embedded:
	application/octet-stream: 100
	image/png: 3
	image/jpeg: 76
	video/quicktime: 10
	image/gif: 1
	text/x-vlocation: 1
	application/vnd.apple.pkpass: 1
	text/vcard: 1
Attachments embedded:
	image/png: 3
	image/jpeg: 76
	image/gif: 1
Attachments missing (see warnings above): 0
HEIC conversions completed: 57
HEIC conversions failed (see warnings above): 0
Time elapsed: 2m32.104904275s

BAGOUP RESULTS:
Export folder: "testmagick"
Export files written: 45
Chats exported: 48
Messages exported: 2154
Attachments referenced or embedded:
	application/vnd.apple.pkpass: 1
	text/vcard: 1
	application/octet-stream: 100
	image/png: 3
	image/jpeg: 76
	video/quicktime: 10
	image/gif: 1
	text/x-vlocation: 1
Attachments embedded:
	image/png: 3
	image/jpeg: 76
	image/gif: 1
Attachments missing (see warnings above): 0
HEIC conversions completed: 57
HEIC conversions failed (see warnings above): 0
Time elapsed: 3m0.292278226s

[tagatac]

In light of the performance advantages of goheif over imagemagick, I'm going to stick with goheif. I migrated to tagatac/goheif in 1b3e86c, which achieves goals 2 and 3 from the OP.

There is probably still room for stability improvement by either

• compiling libheif into bagoup, or
• reimplement relevant portions of libheif in bagoup

PRs welcome

[MikeRich88]

What about the sips command? It's only on Mac OS, but it's also guaranteed to exist.

It only supports HEIC on whatever OS version added HEIC though - but then it's unlikely* you'd have messages with HEIC images on an OS before that.

*Messages is stupid and I've gotten HEIC's before in this manner.

[tagatac]

I did not know about this command! When I created this issue, I was still trying to keep bagoup consistent across MacOS and Linux. However, they've since diverged when I added an Objective C program to parse the contents of the attributedBody column in the message table. It's definitely worth investigating the stability and performance of sips. Thanks for calling it out!