bing search error. #133
Comments
|
Seems like it's not the issue with bing search, as bing search only returns top ranked URLs. The web page extraction (knowledge_storm/utils.py, urls_to_articles function) handles content parsing. Most likely due to network issues. Maybe try another network / WiFI, configure VPN if needed. |
|
I'm experiencing the same thing. Setting retriever to You works but Bing throws many 403s. |
|
You.com api directly return web page content, while bing search returns urls followed by web scrapping. Will close this issue if no further question is posted by end of this week. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
Bing search cant generate report and many 403 error even there is working site.
_run_conversation
conv = future.result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 259, in run_conv
return conv_simulator(
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/dspy/primitives/program.py", line 26, in call
return self.forward(*args, **kwargs)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 55, in forward
expert_output = self.topic_expert(topic=topic, question=user_utterance, ground_truth_url=ground_truth_url)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/dspy/primitives/program.py", line 26, in call
return self.forward(*args, **kwargs)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/knowledge_storm/storm_wiki/modules/knowledge_curation.py", line 174, in forward
searched_results: List[StormInformation] = self.retriever.retrieve(list(set(queries)),
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/knowledge_storm/storm_wiki/modules/retriever.py", line 244, in retrieve
retrieved_data_list = self._rm(query_or_queries=query, exclude_urls=exclude_urls)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/dspy/retrieve/retrieve.py", line 30, in call
return self.forward(*args, **kwargs)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/knowledge_storm/rm.py", line 158, in forward
valid_url_to_snippets = self.webpage_helper.urls_to_snippets(list(url_to_results.keys()))
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/knowledge_storm/utils.py", line 405, in urls_to_snippets
articles = self.urls_to_articles(urls)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/knowledge_storm/utils.py", line 393, in urls_to_articles
article_text = extract(
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/trafilatura/core.py", line 322, in extract
options = Extractor(
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/trafilatura/settings.py", line 86, in init
self._set_format(output_format)
File "/home/bc/Projects/ODS/stormv2/venvStormv2/lib/python3.10/site-packages/trafilatura/settings.py", line 112, in _set_format
raise AttributeError(f"Cannot set format, must be one of: {', '.join(sorted(SUPPORTED_FORMATS))}")
AttributeError: Cannot set format, must be one of: csv, html, json, markdown, python, txt, xml, xmltei
The text was updated successfully, but these errors were encountered: