feat: binary coverage #45
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Retrieve coverage from instrumented `manifestd`
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #45 +/- ##
===========================================
+ Coverage 44.06% 59.20% +15.13%
===========================================
Files 34 34
Lines 2115 2140 +25
===========================================
+ Hits 932 1267 +335
+ Misses 1139 808 -331
- Partials 44 65 +21 ☔ View full report in Codecov by Sentry. |
fmorency
commented
Apr 18, 2024
Comment on lines
+25
to
+27
| // Same as ChainNode.HomeDir() but we need it before the chain is created | ||
| // The node volume is always mounted at /var/cosmos-chain/[chain-name] | ||
| // This is a hackish way to get the coverage files from the ephemeral containers |
There was a problem hiding this comment.
This is the "hackish" part. I'm curious to know if there's a better way.
Comment on lines
+92
to
+128
| // createAuthzJSON generates a JSON file for an authorization message. | ||
| // This is a copy of interchaintest/chain/cosmos/module_authz.go#createAuthzJSON with the addition of the chain environment variables to the Exec call | ||
| func createAuthzJSON(ctx context.Context, chain *cosmos.CosmosChain, filePath string, genMsgCmd []string) error { | ||
| if !strings.Contains(strings.Join(genMsgCmd, " "), "--generate-only") { | ||
| genMsgCmd = append(genMsgCmd, "--generate-only") | ||
| } | ||
|
|
||
| res, resErr, err := chain.GetNode().Exec(ctx, genMsgCmd, chain.Config().Env) | ||
| if resErr != nil { | ||
| return fmt.Errorf("failed to generate msg: %s", resErr) | ||
| } | ||
| if err != nil { | ||
| return err | ||
| } | ||
|
|
||
| return chain.GetNode().WriteFile(ctx, res, filePath) | ||
| } | ||
|
|
||
| // ExecTx executes a transaction, waits for 2 blocks if successful, then returns the tx hash. | ||
| // This is a copy of interchaintest/chain/cosmos/chain_node.go#ExecTx with the addition of the chain environment variables to the Exec call | ||
| func ExecTx(ctx context.Context, chain *cosmos.CosmosChain, keyName string, command ...string) (string, error) { | ||
| stdout, _, err := chain.GetNode().Exec(ctx, chain.GetNode().TxCommand(keyName, command...), chain.Config().Env) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
| output := cosmos.CosmosTx{} | ||
| err = json.Unmarshal([]byte(stdout), &output) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
| if output.Code != 0 { | ||
| return output.TxHash, fmt.Errorf("transaction failed with code %d: %s", output.Code, output.RawLog) | ||
| } | ||
| if err := testutil.WaitForBlocks(ctx, 2, chain.GetNode()); err != nil { | ||
| return "", err | ||
| } | ||
| return output.TxHash, nil |
There was a problem hiding this comment.
I had to duplicate those from interchaintest to add environment variable context so that GOCOVERDIR behaves properly. The helper functions from interchaintest don't support adding environment variable context.
Comment on lines
+151
to
+158
| // Workaround AuthzExec which doesn't propagate the environment variables to the Exec call | ||
| fileName := "authz.json" | ||
| err = createAuthzJSON(ctx, chain, fileName, nestedCmd) | ||
| require.NoError(t, err) | ||
|
|
||
| _, err = ExecTx(ctx, chain, grantee.KeyName(), []string{"authz", "exec", path.Join(chain.GetNode().HomeDir(), fileName)}...) | ||
| require.Error(t, err) | ||
| require.Contains(t, err.Error(), poa.ErrStakingActionNotAllowed.Error()) | ||
| require.ErrorContains(t, err, poa.ErrStakingActionNotAllowed.Error()) |
There was a problem hiding this comment.
Not using the workaround results in the test failing because GOCOVERDIR is not properly set.
| @@ -83,3 +99,38 @@ func AppEncoding() *sdktestutil.TestEncodingConfig { | |||
|
|
|||
| return &enc | |||
| } | |||
|
|
|||
| func CopyCoverageFromContainer(ctx context.Context, t *testing.T, client *client.Client, containerId string, internalGoCoverDir string) { | |||
There was a problem hiding this comment.
The dockerutil module is not exposed in interchaintest v8.2.0. This workaround works just fine.
There was a problem hiding this comment.
this is fixed in main of ictest, just not a proper release yet
| - name: Run coverage | ||
| run: make coverage | ||
| - name: Upload coverage to Codecov | ||
| uses: codecov/codecov-action@v4-beta | ||
| uses: codecov/codecov-action@v4.3.0 |
There was a problem hiding this comment.
We need that version now or else coverage from fork PR won't get uploaded.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Retrieve coverage from instrumented
manifestdbinary.This PR supports measuring the coverage of an instrumented
manifestdbinary.The
manifestdbinary is used ininterchaintestto perform various operations. This PR enables retrieving themanifestdcoverage information from ephemeral containers.The way of doing so is hackish for lack of a better word. I'm curious to know if there's a better way...
Also,
interchaintesthelpers function, e.g.,AuthzExec,TokenFactoryMintDenom, etc, don't allow passing environment variables. I had to work around it.I should probably decouple the
interchaintestfrom the coverage feature at some point...