[AUTOCUT] Integration Test failed for opensearch-system-templates: 2.17.0

[gaiksaya]

The integration test failed at distribution level for component opensearch-system-templates
Version: 2.17.0
Distribution: tar
Architecture: x64
Platform: linux

Please check the logs: https://build.ci.opensearch.org/job/integ-test/8630/display/redirect

* Test-report manifest:*
- https://ci.opensearch.org/ci/dbc/integ-test/2.17.0/10271/linux/x64/tar/test-results/8630/integ-test/test-report.yml

Note: Steps to reproduce, additional logs and other files can be found within the above test-report manifest.
Instructions of this test-report manifest can be found here.

[gaiksaya]

Hi @mohitamg
Seeing below error in integration tests:

    {"error":{"root_cause":[{"type":"invalid_index_template_exception","reason":"index_template [my-metrics-template] invalid, cause [index template [my-metrics-template] specifies a context which cannot be used without enabling: cluster.application_templates.enabled]"}],"type":"invalid_index_template_exception","reason":"index_template [my-metrics-template] invalid, cause [index template [my-metrics-template] specifies a context which cannot be used without enabling: cluster.application_templates.enabled]"},"status":400}

        at __randomizedtesting.SeedInfo.seed([CEEB7C3E9FD3604E:40C1544C45A8D424]:0)

        at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:479)

        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:371)

        at app//org.opensearch.client.RestClient.performRequest(RestClient.java:346)

        at app//org.opensearch.system.applicationtemplates.CreateIndexTemplateWithContextTemplateIT.testCreateIndexWithContextBasedTemplate(CreateIndexTemplateWithContextTemplateIT.java:59)

  2> NOTE: leaving temporary files on disk at: /tmp/tmppekdv1i0/opensearch-system-templates/build/testrun/integTest/temp/org.opensearch.system.applicationtemplates.CreateIndexTemplateWithContextTemplateIT_CEEB7C3E9FD3604E-001

  2> NOTE: test params are: codec=Asserting(Lucene99): {}, docValues:{}, maxPointsInLeafNode=177, maxMBSortInHeap=7.269244417062377, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=be-BY, timezone=ECT

  2> NOTE: Linux 6.1.102-108.177.amzn2023.x86_64 amd64/Eclipse Adoptium 21.0.1 (64-bit)/cpus=4,threads=2,free=316366808,total=536870912

  2> NOTE: All tests run in this JVM: [CreateIndexTemplateWithContextTemplateIT]

Do we need to add any additional cluster config?

[mgodwan]

Yes, the tests require 2 settings to be set.

We do set this in build.gradle

opensearch-system-templates/build.gradle

Lines 230 to 231 in d71e125

	systemProperty 'opensearch.experimental.feature.application_templates.enabled', 'true'
	setting 'cluster.application_templates.enabled', 'true'

Are these not honored in CI run or is the testClusters.integTest being overridden?

[gaiksaya]

No! Since we are testing against the remote cluster deployed in localhost not the internal JVM spun cluster. See the command used here: https://github.com/opensearch-project/opensearch-build/blob/main/scripts/default/integtest.sh#L105

[gaiksaya]

We would need to add something like this in the test manifest https://github.com/opensearch-project/opensearch-build/blob/main/manifests/2.17.0/opensearch-2.17.0-test.yml#L137-L139
Can you create a PR for this component?

[mgodwan]

Got it! Thanks for the pointer
Let me raise a PR to unblock this

[mgodwan]

Raised the PR : opensearch-project/opensearch-build#5002

[gaiksaya]

Closing the issue as tests are passing now. https://build.ci.opensearch.org/blue/organizations/jenkins/integ-test/detail/integ-test/8631/pipeline/
Should re-open in case it fails for other platforms/archs.

Thanks!