Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uninitialized constant OpenSearch::Transport::Transport::HTTP::Faraday::VERSION #115

Closed
1 of 2 tasks
paraiso-angelojohn opened this issue Sep 18, 2023 · 4 comments
Closed
1 of 2 tasks

Comments

@paraiso-angelojohn
Copy link

(check apply)

  • read the contribution guideline
  • (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.

Steps to replicate

Done by installing plugin via Docker build

excerpt from Dockerfile

FROM fluent/fluentd:v1.16.2-debian-1.0
... <some lines here> ...
gem install fluent-plugin-opensearch -v 1.0.8
... <some lines here> ...

I noticed that suddenly, building v1.0.8 of the plugin had some changes in the gems being installed as "dependencies"

Previously (07/27/2023):

Successfully installed multi_json-1.15.0
Successfully installed faraday-retry-1.0.3
Successfully installed faraday-rack-1.0.0
Successfully installed faraday-patron-1.0.0
Successfully installed faraday-net_http_persistent-1.2.0
Successfully installed faraday-net_http-1.0.1
Successfully installed multipart-post-2.3.0
Successfully installed faraday-multipart-1.0.4
Successfully installed faraday-httpclient-1.0.1
Successfully installed faraday-excon-1.1.0
Successfully installed faraday-em_synchrony-1.0.0
Successfully installed faraday-em_http-1.0.0
Successfully installed faraday-1.10.3
Successfully installed opensearch-transport-2.1.0
Successfully installed opensearch-api-2.2.0
Successfully installed opensearch-ruby-2.1.0
Successfully installed aws-eventstream-1.2.0
Successfully installed aws-sigv4-1.6.0
Successfully installed faraday_middleware-aws-sigv4-0.6.1
Successfully installed excon-0.100.0
Successfully installed jmespath-1.6.2
Successfully installed aws-partitions-1.793.0
Successfully installed aws-sdk-core-3.180.0
Successfully installed fluent-plugin-opensearch-1.0.8
24 gems installed

Now (09/18/2023):

Successfully installed multi_json-1.15.0
Successfully installed faraday-retry-1.0.3
Successfully installed faraday-rack-1.0.0
Successfully installed faraday-patron-1.0.0
Successfully installed faraday-net_http_persistent-1.2.0
Successfully installed faraday-net_http-1.0.1
Successfully installed multipart-post-2.3.0
Successfully installed faraday-multipart-1.0.4
Successfully installed faraday-httpclient-1.0.1
Successfully installed faraday-excon-1.1.0
Successfully installed faraday-em_synchrony-1.0.0
Successfully installed faraday-em_http-1.0.0
Successfully installed faraday-1.10.3
Successfully installed opensearch-ruby-3.0.0
Successfully installed aws-eventstream-1.2.0
Successfully installed aws-sigv4-1.6.0
Successfully installed faraday_middleware-aws-sigv4-0.6.1
Successfully installed excon-0.103.0
Successfully installed jmespath-1.6.2
Successfully installed aws-partitions-1.824.0
Successfully installed aws-sdk-core-3.181.1
Successfully installed fluent-plugin-opensearch-1.0.8
22 gems installed

Now when I deploy the newly built image, I get below error message during Fluentd startup:

2023-09-18 10:22:16 +0000 [error]: #0 unexpected error error_class=NameError error="uninitialized constant OpenSearch::Transport::Transport::HTTP::Faraday::VERSION\n\n              \"opensearch-ruby/\#{VERSION} (\#{meta.join('; ')})\"\n                                 ^^^^^^^"
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/opensearch-ruby-3.0.0/lib/opensearch/transport/transport/http/faraday.rb:94:in `user_agent_header'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/opensearch-ruby-3.0.0/lib/opensearch/transport/transport/base.rb:425:in `apply_headers'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/opensearch-ruby-3.0.0/lib/opensearch/transport/transport/http/faraday.rb:73:in `__build_connection'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/opensearch-ruby-3.0.0/lib/opensearch/transport/transport/base.rb:168:in `block in __build_connections'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/opensearch-ruby-3.0.0/lib/opensearch/transport/transport/base.rb:159:in `map'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/opensearch-ruby-3.0.0/lib/opensearch/transport/transport/base.rb:159:in `__build_connections'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/opensearch-ruby-3.0.0/lib/opensearch/transport/transport/base.rb:68:in `initialize'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-opensearch-1.0.8/lib/fluent/plugin/out_opensearch.rb:650:in `new'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-opensearch-1.0.8/lib/fluent/plugin/out_opensearch.rb:650:in `client'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-opensearch-1.0.8/lib/fluent/plugin/opensearch_index_template.rb:62:in `host_unreachable_exceptions'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-opensearch-1.0.8/lib/fluent/plugin/opensearch_index_template.rb:71:in `rescue in retry_operate'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-opensearch-1.0.8/lib/fluent/plugin/opensearch_index_template.rb:69:in `retry_operate'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-opensearch-1.0.8/lib/fluent/plugin/out_opensearch.rb:521:in `handle_last_seen_os_major_version'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluent-plugin-opensearch-1.0.8/lib/fluent/plugin/out_opensearch.rb:400:in `configure'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin.rb:187:in `configure'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/agent.rb:132:in `add_match'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/agent.rb:74:in `block in configure'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/agent.rb:64:in `each'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/agent.rb:64:in `configure'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:149:in `configure'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:105:in `configure'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:80:in `run_configure'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:616:in `block in run_worker'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2023-09-18 10:22:16 +0000 [error]: #0 <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2023-09-18 10:22:16 +0000 [error]: #0 <internal:/usr/local/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:25:in `load'
  2023-09-18 10:22:16 +0000 [error]: #0 /usr/local/bundle/bin/fluentd:25:in `<main>'

When I tried using the image I built on 07/27/2023, I am able to use the plugin successfully - logs are pushed to Opensearch

Provide example config and message

<source>
  @type tail
  path {{ path_inside_docker }}/gc_stats.*
  exclude_path ["{{ path_inside_docker }}/*.gz"]
  pos_file {{ fluentd_pos_file_path }}/gc_stats.pos
  read_from_head true
  follow_inodes true
  tag gc_stats
  <parse>
    @type multiline_grok
    multiline_start_regexp /\d{4}-\d{2}-\d{2}/
    <grok>
      pattern %{TIMESTAMP_ISO8601:@timestamp}\:\s%{GREEDYDATA:message}
      time_key @timestamp
      time_format %Y-%m-%dT%H:%M:%S.%L
    </grok>
  </parse>
</source>

<filter gc_stats>
  @type record_transformer
  <record>
    application "Solr"
    environment {{ deploy_env }}
    tenant_id {{ tenant_id }}
    hostname ${hostname}
    logs_path "{{ solr_logs_dir }}/gc_stats.*"
  </record>
</filter>

<match gc_stats>
  @type opensearch
  port 443
  user {{ opensearch_admin_username }}
  password {{ opensearch_admin_password }}
  logstash_format true
  logstash_prefix {{ app_log_index_name }}
  logstash_dateformat %Y.%m.%d
  utc_index false
  request_timeout {{ fluentd_conf_timeout }}

  <buffer tag,time>
    timekey 1h
    timekey_wait 5m
    flush_mode interval
    flush_interval 30s
    retry_type exponential_backoff
    retry_max_interval 10
  </buffer>

  <endpoint>
    url https://{{ opensearch_domain_endpoint }}
    region {{ opensearch_region }}
  </endpoint>
</match>

Expected Behavior or What you need to ask

Is there a new required configuration that I have not included in my current config - which I am surprise that I need to update since I did not update my Opensearch plugin version.

Using Fluentd and OpenSearch plugin versions

  • OS version
    Docker - image tag v1.16.2-debian-1.0
  • Bare Metal or within Docker or Kubernetes or others?
    Docker
  • Fluentd v1.0 or later
    v1.16.2-debian-1.0
  • OpenSearch plugin version
    1.0.8
  • OpenSearch version (optional)
    2.3
  • OpenSearch template(s) (optional)
@rajathvan
Copy link

I'm having the same issue, is there any fix that you found?

@paraiso-angelojohn
Copy link
Author

@rajathvan looks like related to below issue on Opensearch Ruby side

opensearch-project/opensearch-ruby#205

@paraiso-angelojohn
Copy link
Author

Sharing here my temporary workaround - create a Gemfile containing the known working versions of gems before opensearch-ruby-3.0.0 has been released.

source 'https://rubygems.org'
gem 'multi_json', '1.15.0'
gem 'faraday-retry', '1.0.3'
gem 'faraday-rack', '1.0.0'
gem 'faraday-patron', '1.0.0'
gem 'faraday-net_http_persistent', '1.2.0'
gem 'faraday-net_http', '1.0.1'
gem 'multipart-post', '2.3.0'
gem 'faraday-multipart', '1.0.4'
gem 'faraday-httpclient', '1.0.1'
gem 'faraday-excon', '1.1.0'
gem 'faraday-em_synchrony', '1.0.0'
gem 'faraday-em_http', '1.0.0'
gem 'faraday', '1.10.3'
gem 'opensearch-transport', '2.1.0'
gem 'opensearch-api', '2.2.0'
gem 'opensearch-ruby', '2.1.0'
gem 'aws-eventstream', '1.2.0'
gem 'aws-sigv4', '1.6.0'
gem 'faraday_middleware-aws-sigv4', '0.6.1'
gem 'excon', '0.100.0'
gem 'jmespath', '1.6.2'
gem 'aws-partitions', '1.793.0'
gem 'aws-sdk-core', '3.180.0'
gem 'fluent-plugin-opensearch', '1.0.8'

@24367dfa
Copy link
Contributor

Should the dependency opensearch-ruby be pinnend in fluent-plugin-opensearch.gemspec to avoid Problems like this in the future? It seems recently opensearch-ruby switched to semantic versioning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants