Unhandled errors

[dgw]

For example, rate limits manifest like so (deeply nested path to site-packages elided for readability):

[2024-10-08 03:42:09,484] sopel.bot            ERROR    - Unexpected RatelimitException (https://duckduckgo.com/ 202 Ratelimit) from dgw. Message was: .g jar-jar binks how rude
Traceback (most recent call last):
  File "/.../sopel/bot.py", line 695, in call_rule
    rule.execute(sopel, trigger)
  File "/.../sopel/plugins/rules.py", line 1274, in execute
    exit_code = self._handler(bot, trigger)
  File "/.../sopel_search/plugin.py", line 47, in search
    results = bot.memory['ddg_search_client'].text(
  File "/.../duckduckgo_search/duckduckgo_search.py", line 239, in text
    results = self._text_api(keywords, region, safesearch, timelimit, max_results)
  File "/.../duckduckgo_search/duckduckgo_search.py", line 273, in _text_api
    vqd = self._get_vqd(keywords)
  File "/.../duckduckgo_search/duckduckgo_search.py", line 134, in _get_vqd
    resp_content = self._get_url("POST", "https://duckduckgo.com", data={"q": keywords})
  File "/.../duckduckgo_search/duckduckgo_search.py", line 129, in _get_url
    raise RatelimitException(f"{resp.url} {resp.status_code} Ratelimit")
duckduckgo_search.exceptions.RatelimitException: https://duckduckgo.com/ 202 Ratelimit

This isn't currently handled, so the bot just spits out an "Unexpected RatelimitException" message.

[dgw]

So it's not that simple… duckduckgo_search sets an event flag when it hits an error, and doesn't appear to ever clear that flag.

If the event flag is set, it simply bails on any further attempts:

https://github.com/deedy5/duckduckgo_search/blob/b2b4b43e155afa18114b76c94fab4ce263cfeaa4/duckduckgo_search/duckduckgo_search.py#L116-L117

    def _get_url(
        self,
        method: str,
        url: str,
        params: dict[str, str] | None = None,
        content: bytes | None = None,
        data: dict[str, str] | bytes | None = None,
    ) -> bytes:
        if self._exception_event.is_set():
            raise DuckDuckGoSearchException("Exception occurred in previous call.")

If the event flag is set already by the time anything is raised to the original caller… I'm not sure how code using this library is meant to deal with errors, then. One error and you get shut out until you destroy the DDGS instance, apparently. Guess one is meant to use a fresh DDGS() object each time? Seems inefficient.