[Bug]: When the delegate ends with an error, CodeAct fails too

[enyst]

Is there an existing issue for the same bug?

• I have checked the troubleshooting document at https://docs.all-hands.dev/modules/usage/troubleshooting
• I have checked the existing issues.

Describe the bug

When the delegate is BrowsingAgent and gets stuck in a loop: the main agent gets control, but immediately fails:
...agenthub/codeact_agent/codeact_agent.py", line 169, in get_observation_message raise ValueError(f'Unknown observation type: {type(obs)}') ValueError: Unknown observation type: <class 'openhands.events.observation.browse.BrowserOutputObservation'>

The reason is that the delegate ended with an error, but didn't end with an AgentDelegateObservation. There is no such obs in the log, I'm not sure how it got bypassed or removed. It should happen even if the delegate ended with an error. When an AgentDelegateObservation is added to the event stream and propagated, this would happen: which removes the delegate events from the parent's history. That didn't happen here, so the BrowsingAgent-specific events end up in CodeAct history and are rejected as unknown with a ValueError here.

Display log

22:33:45 - openhands:DEBUG: stream.py:134 - Adding BrowserOutputObservation id=39 from AGENT
22:33:45 - openhands:INFO: session.py:139 - Server event
22:33:45 - openhands:INFO: agent_controller.py:222
OBSERVATION
ErrorObservation(content='Agent got stuck in a loop', observation='error')
22:33:45 - openhands:INFO: session.py:139 - Server event
22:33:45 - openhands:INFO: session.py:139 - Server event
22:33:45 - openhands:INFO: session.py:139 - Server event
22:33:45 - openhands:INFO: agent_controller.py:222
OBSERVATION
ErrorObservation(content='Delegator agent encounters an error', observation='error')
22:33:45 - openhands:INFO: session.py:139 - Server event
22:33:45 - openhands:INFO: session.py:139 - Server event

==============
CodeActAgent LEVEL 0 LOCAL STEP 5 GLOBAL STEP 10

Traceback (most recent call last):
File "/Users/enyst/repos/odie/openhands/controller/agent_controller.py", line 153, in _start_step_loop
await self._step()
File "/Users/enyst/repos/odie/openhands/controller/agent_controller.py", line 463, in _step
action = self.agent.step(self.state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/enyst/repos/odie/agenthub/codeact_agent/codeact_agent.py", line 195, in step
messages = self._get_messages(state)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/enyst/repos/odie/agenthub/codeact_agent/codeact_agent.py", line 249, in _get_messages
message = self.get_observation_message(event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/enyst/repos/odie/agenthub/codeact_agent/codeact_agent.py", line 169, in get_observation_message
raise ValueError(f'Unknown observation type: {type(obs)}')
ValueError: Unknown observation type: <class 'openhands.events.observation.browse.BrowserOutputObservation'>
22:33:45 - openhands:ERROR: agent_controller.py:159 - Error while running the agent: Unknown observation type: <class 'openhands.events.observation.browse.BrowserOutputObservation'>
22:33:45 - openhands:ERROR: agent_controller.py:160 - Traceback (most recent call last):
File "/Users/enyst/repos/odie/openhands/controller/agent_controller.py", line 153, in _start_step_loop
await self._step()
File "/Users/enyst/repos/odie/openhands/controller/agent_controller.py", line 463, in _step
action = self.agent.step(self.state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/enyst/repos/odie/agenthub/codeact_agent/codeact_agent.py", line 195, in step
messages = self._get_messages(state)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/enyst/repos/odie/agenthub/codeact_agent/codeact_agent.py", line 249, in _get_messages
message = self.get_observation_message(event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/enyst/repos/odie/agenthub/codeact_agent/codeact_agent.py", line 169, in get_observation_message
raise ValueError(f'Unknown observation type: {type(obs)}')
ValueError: Unknown observation type: <class 'openhands.events.observation.browse.BrowserOutputObservation'>

22:33:45 - openhands:DEBUG: stream.py:134 - Adding ErrorObservation id=40 from AGENT
22:33:45 - openhands:DEBUG: agent_controller.py:238 - [Agent Controller 22d6796e-91d8-4029-8671-bafdfb37610a] Setting agent(CodeActAgent) state from AgentState.RUNNING to AgentState.ERROR
22:33:45 - openhands:DEBUG: stream.py:134 - Adding AgentStateChangedObservation id=41 from AGENT
22:33:45 - openhands:INFO: agent_controller.py:222
OBSERVATION
ErrorObservation(content='There was an unexpected error while running the agent', observation='error')

Current OpenHands version

main, commit ecf4aed28b0cf7c18d4d8ff554883ba182fc6bdd

Installation and Configuration

make build
make run

Model and Agent

• Model: groq/llama3-8b-8192
• Agent: CodeActAgent

Operating System

Mac OS

Reproduction Steps

It's kind of easy with llama on groq because it doesn't work very well: it started to create and test the app described in our example in the prompt, the BrowsingAgent got errors and got stuck in a loop, then CodeAct was unable to continue.
To reproduce, you want to provoke an error in the delegate. I'm not sure if other errors than stuck in a loop lead to the same, it could matter. Because they might be handled differently, on stuck the execution gets out of the controller step immediately.

Logs, Errors, Screenshots, and Additional Context

No response