feat/add sources from unstructured inference

[benjats07]

This PR adds support for source property from unstructured_inference, allowing the user to be able to see the origin of the data under detection_originfield environment variable UNSTRUCTURED_INCLUDE_DEBUG_METADATA=true

In order to try this feature you can use this code:

from unstructured.partition.pdf import partition_pdf_or_image

yolox_elements = partition_pdf_or_image(filename='example-docs/loremipsum-flat.pdf', strategy='hi_res', model_name='yolox')

sources = [e.detection_origin for e in yolox_elements]
print(sources)

And will print 'yolox' as source for all the elements

[cragwolfe]

some quick feedback. this is mainly for debugging purposes, right?

• let's avoid overloading the word "source" which is has a meaning for data connectors.
• assuming this "breadcrumb" exists for debugging purposes, it should not exist in any element json output by default

[benjats07]

some quick feedback. this is mainly for debugging purposes, right?

* let's avoid overloading the word "source" which is has a meaning for data connectors.

* assuming this "breadcrumb" exists for debugging purposes, it should not exist in any element json output by default

• "source" isn't used anymore in this PR
• Restored original output

[qued]

LGTM!

changes addressed

[badGarnet]

maybe use mocker to set the constant to true so we are certain this is tested; I understand that in the ci we set the env variable so this is tested in CI but this if statement here can cause confusion for local testing vs. ci; and potentially have code silently fail if ci for some reason dropped the env

[badGarnet]

here and below where we hard wire "detection_origin" maybe we can use a parameter to hold those debug_metadata_keys for checks so it is easier to change them later (just change one place)

[benjats07]

Probably, but at this point, I prefer not doing that as this is the only parameter with that behavior, I can add it if we begin to add variables just during debugging