[builder] Support for --skip-new-go-module #10098

kristinapathak · 2024-05-07T03:41:57Z

A continuation of #9253 and #9631

Description: Adds a --skip-new-go-module flag to the OTC builder. This enables users working in an existing go module environment (say, a "monorepo") to update the module they have, vs forcing the use of a new module.

With the new support inside an existing Go module, a collector main package can be generated using a go:generate directive. For example, in the directory where I want my collector built, the file generate.go has this line:

//go:generate builder --skip-new-go-module --skip-compilation --strict-versioning --config=./build-config.yaml
In the same directory, the build-config.yaml describes the collector to build. The builder generates the other files in the same directory. At this point, normal Go workflows can be used to update indirect dependencies.

Link to tracking Issue: #9252

Testing: Will add unit tests in the next few days.

Documentation: Yes.

linux-foundation-easycla · 2024-05-07T03:42:01Z

The committers listed above are authorized under a signed CLA.

✅ login: kristinapathak / name: Kristina Pathak (a910c2d, d375dcb, ad68568, 56ec03b)

codecov · 2024-05-07T03:52:52Z

Codecov Report

Attention: Patch coverage is 96.49123% with 2 lines in your changes missing coverage. Please review.

Project coverage is 91.80%. Comparing base (959e7bc) to head (ad68568).
Report is 4 commits behind head on main.

Files	Patch %	Lines
cmd/builder/internal/builder/main.go	95.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10098      +/-   ##
==========================================
+ Coverage   91.62%   91.80%   +0.18%     
==========================================
  Files         406      406              
  Lines       19046    19080      +34     
==========================================
+ Hits        17450    17516      +66     
+ Misses       1237     1216      -21     
+ Partials      359      348      -11

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

cmd/builder/internal/builder/config.go

cmd/builder/README.md

cmd/builder/internal/builder/main.go

cmd/builder/internal/command.go

github-actions · 2024-05-24T03:15:50Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

mx-psi

Is there any chance to increase testing coverage here? Some error cases would be harder to reach, but there are other parts that should be doable

kristinapathak · 2024-05-27T04:57:53Z

@mx-psi, sorry for the delay. I hit this issue when adding back tests and then got pulled into other things:

failed to update go.mod: go subcommand failed with args '[mod tidy -compat=1.21]': exit status 1, error message: go: finding module for package go.opentelemetry.io/collector/internal/testdata
        	            	go: finding module for package go.opentelemetry.io/collector/confmap/converter/expandconverter
        	            	go: found go.opentelemetry.io/collector/confmap/converter/expandconverter in go.opentelemetry.io/collector/confmap/converter/expandconverter v0.101.0
        	            	go: finding module for package go.opentelemetry.io/collector/internal/testdata
        	            	go: go.opentelemetry.io/collector/cmd/builder/internal/tester/output imports
        	            		go.opentelemetry.io/collector/connector tested by
        	            		go.opentelemetry.io/collector/connector.test imports
        	            		go.opentelemetry.io/collector/internal/testdata: module go.opentelemetry.io/collector@latest found (v0.101.0), but does not contain package go.opentelemetry.io/collector/internal/testdata

I'll try to fix this and increase coverage more this week.

github-actions · 2024-06-11T03:16:00Z

This PR was marked stale due to lack of activity. It will be closed in 14 days.

mx-psi · 2024-06-13T08:29:41Z

@kristinapathak feel free to ping me when tests are passing :)

kristinapathak · 2024-06-27T21:41:45Z

cmd/builder/internal/builder/main.go

@@ -144,7 +155,7 @@ func GetModules(cfg Config) error {
 	}


go mod tidy is run a few lines above and downloads the needed dependencies, so go mod download isn't needed. No dependencies are updated after the tidy call.

Thanks, this comment was useful!

kristinapathak · 2024-06-28T03:05:53Z

cmd/builder/internal/builder/main.go

+	updatespec := "-require=" + mod + "@" + ver
+
+	if _, err := runGoCommand(*c, "mod", "edit", updatespec); err != nil {
+		return err


I couldn't figure out how to get go mod edit to fail 😞 everything I tried caused other failures...

mx-psi · 2024-07-18T12:30:52Z

CI failures are due to #10661, will merge this PR after merging that one

mx-psi · 2024-07-18T14:07:23Z

--- FAIL: TestGetModules (36.26s)
    --- FAIL: TestGetModules/Exporter_Not_in_Gomod (3.57s)
        main_test.go:633: 
            	Error Trace:	D:/a/opentelemetry-collector/opentelemetry-collector/cmd/builder/internal/builder/main_test.go:633
            	Error:      	Received unexpected error:
            	            	failed to compile the OpenTelemetry Collector distribution: go subcommand failed with args '[build -trimpath -o  -ldflags=-s -w]': exit status 1, error message: # go.opentelemetry.io/collector/config/configgrpc
Error:             	            	C:\Users\runneradmin\go\pkg\mod\go.opentelemetry.io\collector\config\[email protected]\configgrpc.go:274:62: not enough arguments in call to gcs.Auth.GetClientAuthenticator
            	            		have (map[component.ID]component.Component)
            	            		want (context.Context, map[component.ID]component.Component)

kristinapathak · 2024-07-31T16:24:26Z

@mx-psi, can this be merged?

mx-psi · 2024-07-31T16:35:06Z

@kristinapathak Sorry, should have said something explicitly, CI is not passing so I can't merge (it's a required CI so I literally can't click the button 😄). This error is still happening #10098 (comment)

mx-psi · 2024-08-06T08:26:06Z

Looks like CI is still having trouble

kristinapathak · 2024-08-12T21:42:00Z

@mx-psi, tests are back to passing 🙂

mx-psi · 2024-08-13T09:52:45Z

Looks like there is one last test that is timing out on Windows:

panic: test timed out after 4m0s
running tests:
	TestGetModules (17s)
	TestGetModules/Exporter_Not_in_Gomod (3s)

I am fine with increasing the timeout value. Could you go ahead and do that?

mx-psi · 2024-08-14T10:49:02Z

Looks like bumping the timeout revealed an error:

	Error:      	Error "component dependency not found in go mod file: 'go.opentelemetry.io/collector/confmap/provider/envprovider'. Use --skip-strict-versioning to temporarily disable this check. This flag will be removed in a future minor version" does not contain "failed to generate source file"

mx-psi · 2024-08-16T08:52:45Z

🤔

Another error

--- FAIL: TestGenerateAndCompile (74.27s)
    --- FAIL: TestGenerateAndCompile/No_Dir_Permissions (2.51s)
        main_test.go:441: 
            	Error Trace:	D:/a/opentelemetry-collector/opentelemetry-collector/cmd/builder/internal/builder/main_test.go:441
            	Error:      	An error is expected but got nil.
            	Test:       	TestGenerateAndCompile/No_Dir_Permissions

Co-authored-by: Pablo Baeyens <[email protected]>

mx-psi · 2024-08-22T08:53:03Z

🎉 🎉

)" This reverts commit a0331ed.

This reverts commit a0331ed.

kristinapathak mentioned this pull request May 7, 2024

[builder] Support for --skip-new-go-module #9631

Closed

mx-psi reviewed May 7, 2024

View reviewed changes

kristinapathak force-pushed the builder-skip-go-mod branch from 6b98f5f to 3e69723 Compare May 8, 2024 18:08

github-actions bot added the Stale label May 24, 2024

mx-psi reviewed May 24, 2024

View reviewed changes

github-actions bot removed the Stale label May 25, 2024

kristinapathak force-pushed the builder-skip-go-mod branch from 3b91ace to b901f03 Compare May 27, 2024 04:56

github-actions bot added the Stale label Jun 11, 2024

kristinapathak force-pushed the builder-skip-go-mod branch from b901f03 to 8841297 Compare June 12, 2024 21:09

github-actions bot removed the Stale label Jun 13, 2024

kristinapathak force-pushed the builder-skip-go-mod branch 4 times, most recently from 55cb39f to a3cbda6 Compare June 27, 2024 21:38

kristinapathak commented Jun 27, 2024

View reviewed changes

kristinapathak force-pushed the builder-skip-go-mod branch 3 times, most recently from 9a37a4c to 893f878 Compare June 28, 2024 02:42

kristinapathak commented Jun 28, 2024

View reviewed changes

kristinapathak force-pushed the builder-skip-go-mod branch from 893f878 to 9eb3ee2 Compare June 28, 2024 03:06

kristinapathak marked this pull request as ready for review June 28, 2024 03:06

kristinapathak requested review from a team and atoulme June 28, 2024 03:06

kristinapathak requested a review from mx-psi July 8, 2024 18:19

kristinapathak force-pushed the builder-skip-go-mod branch 2 times, most recently from 5de8cc9 to ac8a09b Compare July 8, 2024 19:44

mx-psi approved these changes Jul 18, 2024

View reviewed changes

kristinapathak force-pushed the builder-skip-go-mod branch from 67ce270 to db19615 Compare July 18, 2024 18:13

kristinapathak force-pushed the builder-skip-go-mod branch from db19615 to 74e3f33 Compare August 6, 2024 04:37

kristinapathak force-pushed the builder-skip-go-mod branch from bb1c117 to ed588d4 Compare August 12, 2024 21:16

kristinapathak and others added 2 commits August 21, 2024 09:53

[builder] Support for --skip-new-go-module

a910c2d

Apply suggestions from code review

d375dcb

Co-authored-by: Pablo Baeyens <[email protected]>

kristinapathak force-pushed the builder-skip-go-mod branch 2 times, most recently from 15b11a2 to e888fd3 Compare August 21, 2024 17:08

add some tests

56ec03b

kristinapathak force-pushed the builder-skip-go-mod branch from e888fd3 to 56ec03b Compare August 21, 2024 17:47

skip file permission test on windows

ad68568

kristinapathak force-pushed the builder-skip-go-mod branch from 0954dd4 to ad68568 Compare August 21, 2024 18:23

mx-psi merged commit a0331ed into open-telemetry:main Aug 22, 2024
49 checks passed

github-actions bot added this to the next release milestone Aug 22, 2024

kristinapathak deleted the builder-skip-go-mod branch August 22, 2024 17:27

codeboten mentioned this pull request Aug 27, 2024

[cmd/builder] test error when creating new release #10975

Open

codeboten added a commit to codeboten/opentelemetry-collector that referenced this pull request Aug 27, 2024

Revert "[builder] Support for --skip-new-go-module (open-telemetry#10098

a24d56c

)" This reverts commit a0331ed.

codeboten added a commit that referenced this pull request Aug 27, 2024

Revert "[builder] Support for --skip-new-go-module (#10098)" (#10978)

79bef21

This reverts commit a0331ed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[builder] Support for --skip-new-go-module #10098

[builder] Support for --skip-new-go-module #10098

kristinapathak commented May 7, 2024

linux-foundation-easycla bot commented May 7, 2024 •

edited

Loading

codecov bot commented May 7, 2024 •

edited

Loading

github-actions bot commented May 24, 2024

mx-psi left a comment

kristinapathak commented May 27, 2024

github-actions bot commented Jun 11, 2024

mx-psi commented Jun 13, 2024

kristinapathak Jun 27, 2024

mx-psi Jul 18, 2024

kristinapathak Jun 28, 2024

mx-psi commented Jul 18, 2024

mx-psi commented Jul 18, 2024

kristinapathak commented Jul 31, 2024

mx-psi commented Jul 31, 2024

mx-psi commented Aug 6, 2024

kristinapathak commented Aug 12, 2024

mx-psi commented Aug 13, 2024

mx-psi commented Aug 14, 2024

mx-psi commented Aug 16, 2024

mx-psi commented Aug 22, 2024

[builder] Support for --skip-new-go-module #10098

[builder] Support for --skip-new-go-module #10098

Conversation

kristinapathak commented May 7, 2024

linux-foundation-easycla bot commented May 7, 2024 • edited Loading

codecov bot commented May 7, 2024 • edited Loading

Codecov Report

github-actions bot commented May 24, 2024

mx-psi left a comment

Choose a reason for hiding this comment

kristinapathak commented May 27, 2024

github-actions bot commented Jun 11, 2024

mx-psi commented Jun 13, 2024

kristinapathak Jun 27, 2024

Choose a reason for hiding this comment

mx-psi Jul 18, 2024

Choose a reason for hiding this comment

kristinapathak Jun 28, 2024

Choose a reason for hiding this comment

mx-psi commented Jul 18, 2024

mx-psi commented Jul 18, 2024

kristinapathak commented Jul 31, 2024

mx-psi commented Jul 31, 2024

mx-psi commented Aug 6, 2024

kristinapathak commented Aug 12, 2024

mx-psi commented Aug 13, 2024

mx-psi commented Aug 14, 2024

mx-psi commented Aug 16, 2024

mx-psi commented Aug 22, 2024

linux-foundation-easycla bot commented May 7, 2024 •

edited

Loading

codecov bot commented May 7, 2024 •

edited

Loading