Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally pass the Beautiful Soup object to constructors #29

Merged
merged 1 commit into from
Sep 27, 2024

Conversation

biolds
Copy link
Contributor

@biolds biolds commented Sep 25, 2024

Hi!

Thanks for this nice library! I would like to integrate it into the Sosse project, a search engine and crawler I'm writing. In this context, I'm extracting the preview while I already have a Soup object with the web page content. So, here is a small patch that permits passing the Soup object without parsing the page again.

@meyt
Copy link
Owner

meyt commented Sep 25, 2024

Hi @biolds, Thanks for the PR, nice feature. But needs some changes:

  • Add tests
  • Use Union type annotation, we still support Python<3.10

@biolds
Copy link
Contributor Author

biolds commented Sep 26, 2024

Outch, I did not notice I broke so much stuff :)
I added a test, let me know if you expected something different.

@@ -14,13 +17,21 @@


class LinkPreview:
def __init__(self, link: Link, parser: str = PARSER):
def __init__(self, link: Link, parser: Union[str, None] = PARSER, soup: Union[BeautifulSoup, None] = None):
if parser and soup:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@biolds No need to check the arguments twice, just in PreviewBase is enough.

raise Exception(
'Only one of `parser` or `soup` argument must be provided to PreviewBase')

if not parser and not soup:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@biolds This kind of validation is not necessary. bs4 is fine without passing parser, and also the developer will receive proper exception if not specifing the soup.

for key in tout.keys():
assert getattr(preview, key) == tout[key]


def test_bs4_object():
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@biolds Please don't touch other test cases, keep this and add another case for when both parser and soup are set.

@biolds
Copy link
Contributor Author

biolds commented Sep 27, 2024

It's updated.

@meyt meyt merged commit 4e9c3b9 into meyt:master Sep 27, 2024
1 of 2 checks passed
@meyt
Copy link
Owner

meyt commented Sep 27, 2024

Thanks @biolds, v0.11.0 released.

@biolds
Copy link
Contributor Author

biolds commented Sep 27, 2024

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants