Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to start - permission errors in log and securityadmin failing to run.. #10

Closed
tombburnell opened this issue Mar 2, 2023 · 1 comment

Comments

@tombburnell
Copy link

Hi,
The docker-compose opensearch is spitting out a lot of errors and the security admin command is failing - even after waiting for quite a while.

Been using this successfully on another VM for the last year - just migrated to a new machine and created a VM and it's not working.

Many thanks, Tom

-----------logs ------------------

kibana_1 | FATAL Error: EACCES: permission denied, open '/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key'

kibana_1 | {"type":"log","@timestamp":"2023-03-02T01:15:15Z","tags":["fatal","root"],"pid":1,"message":"Error: EACCES: permission denied, open '/usr/share/opensearch-dashboards/config/certificate

opensearch-docker-compose_kibana_1 exited with code 1

os02_1 | Likely root cause: OpenSearchException[Unable to read /usr/share/opensearch/config/certificates/os02/os02.key (/usr/share/opensearch/config/certificates/os02/os02.key). Please make sure this files exists and is readable regarding to permissions. Property: plugins.security.ssl.transport.pemkey_filepath]

kibana_1 |
os02_1 | OpenSearch exited with code 1
os02_1 | Performance analyzer exited with code 143
os01_1 | OpenSearch exited with code 1
os01_1 | Performance analyzer exited with code 143
opensearch-docker-compose_kibana_1 exited with code 1
opensearch-docker-compose_os02_1 exited with code 0
opensearch-docker-compose_os01_1 exited with code 0
os03_1 | WARNING: A terminally deprecated method in java.lang.System has been called
os03_1 | WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.Security (file:/usr/share/opensearch/lib/opensearch-2.2.0.jar)
os03_1 | WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.Security
os03_1 | WARNING: System::setSecurityManager will be removed in a future release
os03_1 | [2023-03-02T01:16:02,777][INFO ][o.o.n.Node ] [os03] version[2.2.0], pid[9], build[tar/b1017fa3b9a1c781d4f34ecee411e0cdf930a515/2022-08-09T02:27:25.256769336Z], OS[Linux/5.4.0-139-generic/amd64], JVM[Eclipse Adoptium/OpenJDK 64-Bit Server VM/17.0.4/17.0.4+8]
os03_1 | [2023-03-02T01:16:02,795][INFO ][o.o.n.Node ] [os03] JVM home [/usr/share/opensearch/jdk], using bundled JDK [true]
os03_1 | [2023-03-02T01:16:02,804][INFO ][o.o.n.Node ] [os03] JVM arguments [-Xshare:auto, -Dopensearch.networkaddress.cache.ttl=60, -Dopensearch.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/tmp/opensearch-9462791360918426412, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Dclk.tck=100, -Djdk.attach.allowAttachSelf=true, -Djava.security.policy=/usr/share/opensearch/config/opensearch-performance-analyzer/opensearch_security.policy, --add-opens=jdk.attach/sun.tools.attach=ALL-UNNAMED, -Dopensearch.cgroups.hierarchy.override=/, -Xms1024m, -Xmx1024m, -XX:MaxDirectMemorySize=536870912, -Dopensearch.path.home=/usr/share/opensearch, -Dopensearch.path.conf=/usr/share/opensearch/config, -Dopensearch.distribution.type=tar, -Dopensearch.bundled_jdk=true]
kibana_1 | {"type":"log","@timestamp":"2023-03-02T01:16:04Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin "visTypeXy" is disabled."}
kibana_1 | {"type":"log","@timestamp":"2023-03-02T01:16:04Z","tags":["info","plugins-service"],"pid":1,"message":"Plugin "wizard" is disabled."}
kibana_1 | {"type":"log","@timestamp":"2023-03-02T01:16:04Z","tags":["warning","config","deprecation"],"pid":1,"message":""cpu.cgroup.path.override" is deprecated and has been replaced by "ops.cGroupOverrides.cpuPath""}
kibana_1 | {"type":"log","@timestamp":"2023-03-02T01:16:04Z","tags":["warning","config","deprecation"],"pid":1,"message":""cpuacct.cgroup.path.override" is deprecated and has been replaced by "ops.cGroupOverrides.cpuAcctPath""}
kibana_1 | {"type":"log","@timestamp":"2023-03-02T01:16:04Z","tags":["fatal","root"],"pid":1,"message":"Error: EACCES: permission denied, open '/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key'\n at Object.openSync (fs.js:498:3)\n at readFileSync (fs.js:394:35)\n at readFile (/usr/share/opensearch-dashboards/src/core/server/http/ssl_config.js:181:31)\n at new SslConfig (/usr/share/opensearch-dashboards/src/core/server/http/ssl_config.js:131:18)\n at new HttpConfig (/usr/share/opensearch-dashboards/src/core/server/http/http_config.js:175:16)\n at MapSubscriber.project (/usr/share/opensearch-dashboards/src/core/server/http/http_service.js:61:177)\n at MapSubscriber._next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/operators/map.js:49:35)\n at MapSubscriber.Subscriber.next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/Subscriber.js:66:18)\n at CombineLatestSubscriber.notifyNext (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/observable/combineLatest.js:97:34)\n at InnerSubscriber._next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/InnerSubscriber.js:28:21)\n at InnerSubscriber.Subscriber.next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/Subscriber.js:66:18)\n at MapSubscriber._next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/operators/map.js:55:26)\n at MapSubscriber.Subscriber.next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/Subscriber.js:66:18)\n at DistinctUntilChangedSubscriber._next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/operators/distinctUntilChanged.js:69:30)\n at DistinctUntilChangedSubscriber.Subscriber.next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/Subscriber.js:66:18)\n at MapSubscriber._next (/usr/share/opensearch-dashboards/node_modules/rxjs/internal/operators/map.js:55:26) {\n errno: -13,\n syscall: 'open',\n code: 'EACCES',\n path: '/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key'\n}"}
kibana_1 |
kibana_1 | FATAL Error: EACCES: permission denied, open '/usr/share/opensearch-dashboards/config/certificates/os-dashboards/os-dashboards.key'
kibana_1 |


$ sudo docker-compose exec os01 bash -c "chmod +x plugins/opensearch-security/tools/securityadmin.sh && bash plugins/opensearch-security/tools/securityadmin.sh -cd config/opensearch-security -icl -nhnv -cacert config/certificates/ca/ca.pem -cert config/certificates/ca/admin.pem -key config/certificates/ca/admin.key -h localhost"


** This tool will be deprecated in the next major release of OpenSearch **
** opensearch-project/security#1755 **


Security Admin v7
Will connect to localhost:9200
ERR: Seems there is no OpenSearch running on localhost:9200 - Will exit


port is open but cant curl to it

$ nc -v localhost 9200
Connection to localhost 9200 port [tcp/*] succeeded!
^C
:/opt/opensearch-docker-compose$ curl -v localhost 9200

  • Trying 127.0.0.1:80...
  • TCP_NODELAY set
  • connect to 127.0.0.1 port 80 failed: Connection refused
  • Failed to connect to localhost port 80: Connection refused
  • Closing connection 0
    curl: (7) Failed to connect to localhost port 80: Connection refused
  • Trying 0.0.35.240:80...
  • TCP_NODELAY set
    ^C
@flavienbwk
Copy link
Owner

Logs say it: your certs don't have the right permissions.

Solution was addressed in this issue : #9 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants