Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Apply inferface change to duplicated code in the tgis layer (#50)
In the upstream code the `init_state()` method was removed from the Outlines logits processor classes but the call to this method was still in our tgis layer causing this error: ``` ERROR 06-19 16:12:57 logs.py:77] generate{input=[b'Hello, I am here to'] prefix_id= adapter_id= input_chars=[19] params=decoding { format: JSON }: 'CFGLogitsProcessor' object has no attribute 'init_state' ERROR 06-19 16:12:57 grpc_server.py:86] Generate failed ERROR 06-19 16:12:57 grpc_server.py:86] Traceback (most recent call last): ERROR 06-19 16:12:57 grpc_server.py:86] File "/opt/vllm/lib/python3.11/site-packages/vllm/entrypoints/grpc/grpc_server.py", line 103, in func_with_log ERROR 06-19 16:12:57 grpc_server.py:86] return await func(*args, **kwargs) ERROR 06-19 16:12:57 grpc_server.py:86] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ERROR 06-19 16:12:57 grpc_server.py:86] File "/opt/vllm/lib/python3.11/site-packages/vllm/entrypoints/grpc/grpc_server.py", line 153, in Generate ERROR 06-19 16:12:57 grpc_server.py:86] sampling_params, deadline = await self._validate_and_convert_params( ERROR 06-19 16:12:57 grpc_server.py:86] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ERROR 06-19 16:12:57 grpc_server.py:86] File "/opt/vllm/lib/python3.11/site-packages/vllm/entrypoints/grpc/grpc_server.py", line 430, in _validate_and_convert_params ERROR 06-19 16:12:57 grpc_server.py:86] await get_outlines_guided_decoding_logits_processor(decoding, ERROR 06-19 16:12:57 grpc_server.py:86] File "/opt/vllm/lib/python3.11/site-packages/vllm/tgis_utils/guided_decoding.py", line 44, in get_outlines_guided_decoding_logits_processor ERROR 06-19 16:12:57 grpc_server.py:86] logits_processor.init_state() ERROR 06-19 16:12:57 grpc_server.py:86] ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ``` --------- Signed-off-by: Max de Bayser <[email protected]>
- Loading branch information