-
Notifications
You must be signed in to change notification settings - Fork 743
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(xlsx): XLSX emits std minified .text_as_html #3558
Conversation
] | ||
|
||
|
||
def test_partition_xlsx_with_find_subtables_False_and_infer_table_structure_False_works(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need a test case for find_subtable=True, infer_table_structure=False
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd actually be inclined not to have any tests like this :) This test boils down to "it doesn't raise for this arg combination", which is an impoverished test at best.
But I found a bug while I was in there on the html-minification and needed a test that triggered that bug so I could confirm my fix and defend against regression.
I think the broader remedy would be to break up partition_xlsx()
a bit, like into a _XlsxPartitioner
that we could write isolated unit tests for, but I didn't want to get into the extra scope.
Anyway, as it turns out, that particular case is handled by test_partition_xlsx_infer_table_structure()
on line 95.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good to me then. I had to look up what these parameters both did and wasn't positive on their interaction. It did cross my mind to add find_subtable
to the docstring, but I assumed you would be against that. The docs might need to be updated to reflect that xlsx uses infer_table_structure
, though, because I did query the ai search on the site and it said "The infer_table_structure parameter is not directly applicable to XLSX files in Unstructured."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think both those have a place in the docstring. Definitely find_subtable
, since it's XLSX-specific. infer_table_structure
too because it's used in the partitioner code.
I've added find_subtable
. infer_table_structure
was already there. Sounds like the AI got that wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
020da98
to
50c53d9
Compare
50c53d9
to
1d9587b
Compare
1d9587b
to
b76b29d
Compare
d5554f2
to
52008d3
Compare
52008d3
to
a8fffa0
Compare
a8fffa0
to
2a596f9
Compare
Summary
Eliminate historical "idiosyncracies" of
table.metadata.text_as_html
HTML introduced bypartition_xlsx()
. Produce minified.text_as_html
consistent with that formed by chunking.Additional Context
.text_as_html
is minified (no extra whitespace or thead, tbody, tfoot elements).table.text
is clean-concatenated-text (CCT) of table.