[TT-1218] update testlistgenerator for ccip (#990)

tools/testlistgenerator/README.md

@@ -5,71 +5,110 @@ This Go script builds a JSON file containing tests to be run for a given product
 ## Usage
 
 ```bash
-go run main.go <output_file_name> <product> <test_regex> <file> <eth_implementation> <docker_images>
+go run main.go -t <test_name> -o <output_file_name> -p <product> -r <test_regex> -f <file> -e <eth_implementation> -d <docker_images> -n <node_label> [-c <chain_ids>] [-w <networks>]
 ```
 
 ### Example
 
 ```bash
-go run main.go 'test_list.json' 'ocr' 'TestOCR.*' './smoke/ocr_test.go' 'besu' 'hyperledger/besu:21.0.0,hyperledger/besu:22.0.0'
+go run main.go -t "emv-test" -o "test_list.json" -p "ocr" -r "TestOCR.*" -f "./smoke/ocr_test.go" -e "besu" -d "hyperledger/besu:21.0.0,hyperledger/besu:22.0.0" -n "ubuntu-latest"
 ```
 
 ## Output
 
 The script generates or updates a JSON file with entries structured as follows:
 
 ```json
-{
-  "tests": [
-    {
-      "product": "ocr",
-      "test_regex": "TestOCR.*",
-      "file": "./smoke/ocr_test.go",
-      "eth_implementation": "besu",
-      "docker_image": "hyperledger/besu:21.0.0"
-    },
-    {
-      "product": "ocr",
-      "test_regex": "TestOCR.*",
-      "file": "./smoke/ocr_test.go",
-      "eth_implementation": "besu",
-      "docker_image": "hyperledger/besu:22.0.0"
-    }
-  ]
-}
+[
+  {
+    "name": "emv-test-01",
+    "os": "ubuntu-latest",
+    "product": "ocr",
+    "eth_implementation": "besu",
+    "docker_image": "hyperledger/besu:21.0.0",
+    "run": "-run 'TestOCR.*' ./smoke/ocr_test.go"
+  },
+  {
+    "name": "emv-test-02",
+    "os": "ubuntu-latest",
+    "product": "ocr",
+    "eth_implementation": "besu",
+    "docker_image": "hyperledger/besu:22.0.0",
+    "run": "-run 'TestOCR.*' ./smoke/ocr_test.go"
+  }
+]
+```
+
+If the script is run with optional `chain_id` flag the output is slightly different:
+
+```bash
+go run main.go -t "emv-test" -o "test_list.json" -p "ocr" -r "TestOCR.*" -f "./smoke/ocr_test.go" -e "besu" -d "hyperledger/besu:21.0.0,hyperledger/besu:22.0.0" -n "ubuntu-latest" -c 1337,2337 -w "mainnet,ropsten"
+```
+
+Output:
+
+```json
+[
+  {
+    "name": "emv-test-01",
+    "os": "ubuntu-latest",
+    "product": "ocr",
+    "eth_implementation": "besu",
+    "docker_image": "1337=hyperledger/besu:21.0.0,2337=hyperledger/besu:21.0.0",
+    "run": "-run 'TestOCR.*' ./smoke/ocr_test.go",
+    "networks": "mainnet,ropsten"
+  },
+  {
+    "name": "emv-test-02",
+    "os": "ubuntu-latest",
+    "product": "ocr",
+    "eth_implementation": "besu",
+    "docker_image": "1337=hyperledger/besu:22.0.0,2337=hyperledger/besu:22.0.0",
+    "run": "-run 'TestOCR.*' ./smoke/ocr_test.go",
+    "networks": "mainnet,ropsten"
+  }
+]
 ```
 
 ## Command Line Arguments
 
-- `<output_file_name>`: The name of the JSON file where the test entries will be stored.
-- `<product>`: The name of the product for which the tests are being generated.
-- `<test_regex>`: The regular expression to match test names.
-- `<file>`: The file path where the tests are defined.
-- `<eth_implementation>`: The name of the Ethereum implementation.
-- `<docker_images>`: A comma-separated list of Docker images to be used.
+- `-t <test_name>`: A prefix for the test name.
+- `-o <output_file_name>`: The name of the JSON file where the test entries will be stored.
+- `-p <product>`: The name of the product for which the tests are being generated.
+- `-r <test_regex>`: The regular expression to match test names.
+- `-f <file>`: The file path where the tests are defined.
+- `-e <eth_implementation>`: The name of the Ethereum implementation.
+- `-d <docker_images>`: A comma-separated list of Docker images to be used.
+- `-n <node_label>`: The node label for the test environment.
+- `-c <chain_ids>`: (Optional) A comma-separated list of chain IDs to associate with each Docker image.
+- `-w <networks>`: (Optional) A comma-separated list of networks.
 
 ## Error Handling
 
 The script will panic and display error messages in the following scenarios:
 
 - Insufficient command line arguments.
-- Empty parameters for output_file_name, product, test_regex, file, eth_implementation, or docker_images.
+- Empty parameters for test_name, output_file_name, product, test_regex, file, eth_implementation, docker_images, or node_label.
 - Invalid Docker image format (should include a version tag).
 - Invalid regular expression for test_regex.
+- Invalid or non-integer chain IDs.
 - Errors in file operations (opening, reading, writing).
 
 ## Detailed Steps
 
-1. **Argument Parsing**: The script expects at least 7 command line arguments. It splits the `<docker_images>` argument into a slice.
-2. **Validation**: It validates the input parameters, ensuring none are empty and the regular expression compiles.
+1. **Argument Parsing**: The script uses Cobra to parse command line arguments.
+2. **Validation**: It validates the input parameters, ensuring none are empty, and the regular expression compiles.
 3. **File Operations**:
    - If the output file exists, it reads and unmarshals the content.
    - If it doesn't exist, it creates a new file.
-4. **Appending Entries**: For each Docker image, it creates a new `OutputEntry` and appends it to the output.
+4. **Appending Entries**: For each Docker image, it creates new `OutputEntry` objects and appends them to the output.
 5. **JSON Marshaling**: It marshals the updated output to JSON and writes it back to the file.
 6. **Completion Message**: Prints a success message indicating the number of tests added.
 
 ## Notes
 
 - Ensure the Docker image names include version tags, e.g., `hyperledger/besu:21.0.0`.
-- The script appends new entries; it does not overwrite existing entries in the output file.
+- The script appends new entries; it does not overwrite existing entries in the output file unless the output file is specified with new test entries.
+- Optional parameters like `chain_ids` and `networks` allow for additional customization of the test entries. If `chain_ids` are provided, they will be included in the Docker image field. If `networks` are provided, they will be included as an additional field in the output JSON.
+
+This update reflects the recent changes to include flags for `chain_ids` and `networks`, and ensures the documentation matches the new argument names and structure.

tools/testlistgenerator/go.mod

@@ -2,10 +2,15 @@ module github.com/smartcontractkit/chainlink-testing-framework/tools/testlistgen
 
 go 1.21.7
 
-require github.com/stretchr/testify v1.9.0
+require (
+	github.com/spf13/cobra v1.8.0
+	github.com/stretchr/testify v1.9.0
+)
 
 require (
 	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/inconshreveable/mousetrap v1.1.0 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
+	github.com/spf13/pflag v1.0.5 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )

tools/testlistgenerator/go.sum

@@ -1,7 +1,15 @@
+github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
+github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/spf13/cobra v1.8.0 h1:7aJaZx1B85qltLMc546zn58BxxfZdR/W22ej9CFoEf0=
+github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyhcho=
+github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
+github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
 github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=