ai-proxy nil error with bedrock misconfiguration

[t-yuki]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.8.0.0

Current Behavior

nil error happen when aws_region is missing.
[kong] init.lua:427 [ai-proxy] /usr/local/share/lua/5.1/kong/llm/proxy/handler.lua:471: attempt to call field 'set_response_transformer_skipped' (a nil value)

Expected Behavior

Error detail on AWS auth config such as error authenticating with cloud-provider, AWS region not specified anywhere

Steps To Reproduce

1. in docker environment

2. with config of a route like plugins:
   - name: ai-proxy
   config:
   route_type: llm/v1/chat
   auth:
   aws_access_key_id: MYKEY
   aws_secret_access_key: MYSECRET
   model:
   provider: bedrock
   name: anthropic.claude-3-haiku-20240307-v1:0
   #options: # NO AWS_REGION CONFIG
   # bedrock: {"aws_region": "ap-northeast-1"}

3. call the route

Anything else?

I think

kong/kong/llm/proxy/handler.lua

Line 461 in c983384

llm_state.set_response_transformer_skipped()

should use disable_ai_proxy_response_transform

[chobits]

hi @t-yuki thanks for report, it seems a function typo here. I think your advice might be right.

• not find the function definition after searching the source code:

xc kong $ grep 'set_response_transformer_skipped' -R kong
kong/llm/proxy/handler.lua:    llm_state.set_response_transformer_skipped()

Could you open a pull request for this? We warmly welcome community users to submit PRs to us, and we are also happy to help with reviews.

[github-actions]

This issue is marked as stale because it has been open for 14 days with no activity.

[github-actions]

Dear contributor,

We are automatically closing this issue because it has not seen any activity for three weeks.
We're sorry that your issue could not be resolved. If any new information comes up that could
help resolving it, please feel free to reopen it.

Your contribution is greatly appreciated!

Please have a look
our pledge to the community
for more information.

Sincerely,
Your Kong Gateway team