feat: allow to set a signature port for tunnel usage

[andreaslang]

Description

Allows to change the port used to sign the AWS request which is causing an issue if accessing an AWS Opensearch instance via a tunnel

If you have an ssh tunnel created this works now (while it would not without signature_port):

host = 'localhost'
port = 10012
region = 'eu-west-1'
service = 'es'
credentials = boto3.Session().get_credentials()
auth = AWSV4SignerAuth(credentials, region, service, signature_port=443)

client = OpenSearch(
    hosts = [{'host': host, 'port': port}],
    http_auth = auth,
    use_ssl = True,
    verify_certs = False,
    ssl_assert_hostname = False,
    ssl_show_warn = False,
    connection_class = RequestsHttpConnection,
    pool_maxsize = 20
)

Issues Resolved

#184

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.58%. Comparing base (c8b04a5) to head (4e3ef98).
Report is 87 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #491      +/-   ##
==========================================
- Coverage   70.92%   70.58%   -0.34%     
==========================================
  Files          81       81              
  Lines        7732     7738       +6     
==========================================
- Hits         5484     5462      -22     
- Misses       2248     2276      +28     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dblock]

This is an interesting workaround, but I really don't think it's the right one. Using a tunnel is already quite suspicious for auth, it feels like MITM. With this change we're basically telling the signer to use a different host/port in some very special case - I am struggling to convince myself it's a good idea.

1. What do other clients do? Do they break the same way?
2. Can we just document this case and use a slightly more custom AWSV4SignerAuth?

[dblock]

I would merge a change that allows to override headers, and specifically to override the host header.

[andreaslang]

I would merge a change that allows to override headers, and specifically to override the host header.

Yes, that is a really good point. It should work the same way if I replace the host header like localhost:10012 > realdomain:443. I will change the PR having an Authentication class specifically for that (e.g. TunneledAWSV4SignerAuth) with the explanation in the docstring. Will likely get to it next Tuesday.

[dblock]

I would merge a change that allows to override headers, and specifically to override the host header.

Yes, that is a really good point. It should work the same way if I replace the host header like localhost:10012 > realdomain:443. I will change the PR having an Authentication class specifically for that (e.g. TunneledAWSV4SignerAuth) with the explanation in the docstring. Will likely get to it next Tuesday.

I actually would prefer as a developer to be able to write something like this:

client = OpenSearch(
    hosts = [{'host': host, 'port': port}],
    http_headers: {
        'Host': 'something.us-west2.aws.bla.bla.bla'
    }
)

The signer implementation should be smart enough to consider this host header.

This seems more generic and future-proof, don't you think?

[dblock]

@andreaslang Are you still interested in adding an http_headers option?

[andreaslang]

Yes, I am. Sorry for the delay. I do have a good excuse that my daughter was born though. Planning to pick this up soon.

…

On Thu, 9 Nov 2023, 23:40 Daniel (dB.) Doubrovkine, < ***@***.***> wrote: @andreaslang <https://github.com/andreaslang> Are you still interested in adding an http_headers option? — Reply to this email directly, view it on GitHub <#491 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZJ2U3FVZSYEZW7CZ7R55LYDVSVDAVCNFSM6AAAAAA4UZOLR6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBUHA2DQMRZGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[saimedhi]

Hello @andreaslang! Hope you're doing great. Could you please consider finishing up this PR when you have a moment? Thank you!