CI: Depend On Releases

[Stebalien]

Done Criteria

A CI check catches if go.mod is being updated with any non-released version. This includes checking direct and transitive dependencies. There should be a way for maintainers to override this check (e.g., PR label or code comment that links to explanation).

Why Important

1. Security / reliability - While there's no guarantee that a released version doesn't have bugs or issues, it seems fair to assume that non-released versions have even more. For example, [SEV1] Nodes stop syncing due to "failed to verify beacon". #12467 was triggered because of a bug in non-released library that lotus was depending on when the latest released version didn't have the bug.
2. (bonus) Faster builds
3. (bonus) Makes Lotus a better citizen when it's imported by other projects.

User/Customer

1. All consumers if it makes Lotus more secure/stable.
2. Maintainers and contributors gain from the faster builds

Notes

1. Our "changelog check" workflow could be used for inspiration. Any case where we are letting in a non-released version, we want it to be intentional and documented.
2. It's clear that a true mechanism is needed. At least of 202409 we have 150+ out of 670+ dependencies using a commit hash rather than a released version. This is something maintainers have wanted to change for 3+ years, but without the mechanism this is alive and well.
3. This issue can be marked as done once we prevent new-non-released versions from sneaking in without intentional approval. It's a separate item to clean up "the sins of the past".

[BigLep]

@Stebalien : thanks for creating. This came out of some recent issues right? If easily available, link to them for context.

[Stebalien]

Nothing filed in issues.

Commits we depend on that aren't on master:

1. go-car v0.1.1-0.20201119040415-11b6074b6d4d
2. go-car/v2 v2.0.0-beta1.0.20210721090610-5a9d1b217d25. We don't actually depend on this directly, but go mod tidy needs it for dependency resolution (and the commit was deleted upstream until I revived and tagged it).

We also have a lot of dependencies on master commits. I'm suggesting that we make sure we depend on releases, not just master, because it's strictly easier to check/maintain:

1. Easy to check in CI and/or a review.
2. Easy to tell if there are breaking changes (semver).
3. Going back and adding releases for random commits on master isn't always possible.
4. We want to depend on released versions when we cut releases, so anyone importing lotus (or any of its dependencies) aren't forced to use non-release versions.

[BigLep]

echo "$(go list -m all | egrep 'v\d+\.\d+\.\d+\-' | wc -l) out of $(go list -m all | wc -l) dependencies use non-releaesd versions"

153 out of 672 dependencies use non-releaesd versions

go list -m all | egrep "v\d+\.\d+\.\d+\-"

List of Lotus dependencies that aren't of a released version.

dmitri.shuralyov.com/app/changes v0.0.0-20180602232624-0a106ad413e3
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9
dmitri.shuralyov.com/html/belt v0.0.0-20180602232347-f7d459c86be0
dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412
dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c
git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999
github.com/AndreasBriese/bbloom v0.0.0-20190825152654-46b345b51c96
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802
github.com/Gurpartap/async v0.0.0-20180927173644-4f7f499dd9ee
github.com/Kubuxu/imtui v0.0.0-20210401140320-41663d68d0fa
github.com/PuerkitoBio/urlesc v0.0.0-20170810143723-de5bf2ad4578
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
github.com/ajstarks/svgo v0.0.0-20211024235047-1546f124cd8b
github.com/alecthomas/jsonschema v0.0.0-20200530073317-71f438968921
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751
github.com/alecthomas/units v0.0.0-20231202071711-9a357b53e9c9
github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6
github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625
github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f
github.com/btcsuite/btcutil v0.0.0-20190425235716-9e5f4b9a998d
github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd
github.com/btcsuite/goleveldb v0.0.0-20160330041536-7834afc9e8cd
github.com/btcsuite/snappy-go v0.0.0-20151229074030-0bdef8d06723
github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792
github.com/chromedp/cdproto v0.0.0-20230802225258-3cf4e6d46a89
github.com/cncf/udpa/go v0.0.0-20191209042840-269d4d468f6f
github.com/cncf/xds/go v0.0.0-20240318125728-8a4994d93e50
github.com/coreos/go-systemd v0.0.0-20181012123002-c6f51f82210d
github.com/crackcomm/go-gitignore v0.0.0-20231225121904-e25f5bc08668
github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c
github.com/detailyang/go-fallocate v0.0.0-20180908115635-432fa640bd2e
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13
github.com/filecoin-project/filecoin-ffi v1.28.0-rc2 => ./extern/filecoin-ffi
github.com/filecoin-project/go-data-transfer/v2 v2.0.0-rc7
github.com/filecoin-project/go-state-types v0.15.0-dev
github.com/flynn/go-shlex v0.0.0-20150515145356-3f9db97f8568
github.com/go-check/check v0.0.0-20180628173108-788fd7840127
github.com/go-gl/glfw v0.0.0-20190409004039-e6da0acd62b1
github.com/go-gl/glfw/v3.3/glfw v0.0.0-20200222043503-6f7a984d4dc4
github.com/go-latex/latex v0.0.0-20231108140139-5c1ce85aa4ea
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572
github.com/goccmack/gocc v0.0.0-20230228185258-2292f9e40198
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
github.com/golang/lint v0.0.0-20180702182130-06c8688daad7
github.com/google/pprof v0.0.0-20240509144519-723abb6459b7
github.com/gregdhill/go-openrpc v0.0.0-20220114144539-ae6f44720487
github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7
github.com/hako/durafmt v0.0.0-20200710122514-c0fb7b4da026
github.com/hannahhoward/cbor-gen-for v0.0.0-20230214144701-5d17c9d5243c
github.com/hannahhoward/go-pubsub v0.0.0-20200423002714-8d62886cc36e
github.com/huin/goutil v0.0.0-20170803182201-1ca381bf3150
github.com/ianlancetaylor/demangle v0.0.0-20240312041847-bd984b5ce465
github.com/icrowley/fake v0.0.0-20180203215853-4178557ae428
github.com/icza/backscanner v0.0.0-20210726202459-ac2ffc679f94
github.com/icza/mighty v0.0.0-20180919140131-cfd07d671de6
github.com/influxdata/influxdb1-client v0.0.0-20200827194710-b269163b24ab
github.com/ipld/go-ipld-prime-proto v0.0.0-20191113031812-e32bd156a1e5
github.com/ipld/go-ipld-prime/storage/bsadapter v0.0.0-20230102063945-1a409dc236dd
github.com/ipsn/go-secp256k1 v0.0.0-20180726113642-9d62b9f0bc52
github.com/jackc/pgerrcode v0.0.0-20240316143900-6e2875d9b438
github.com/jackc/pgservicefile v0.0.0-20231201235250-de7065d80cb9
github.com/jbenet/go-random v0.0.0-20190219211222-123a90aedc0c
github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1
github.com/joeshaw/multierror v0.0.0-20140124173710-69b34d4ec901
github.com/kabukky/httpscerts v0.0.0-20150320125433-617593d7dcb3
github.com/kkdai/bstream v0.0.0-20161212061736-f391b8402d23
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515
github.com/marten-seemann/tcp v0.0.0-20210406111302-dfbc87cc63fd
github.com/mikioh/tcp v0.0.0-20190314235350-803a9b46060c
github.com/mikioh/tcpinfo v0.0.0-20190314235526-30a79bb1804b
github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f
github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86
github.com/neelance/sourcemap v0.0.0-20200213170602-2833bce08e4c
github.com/open-rpc/meta-schema v0.0.0-20201029221707-1b72ef2ea333
github.com/opentracing-contrib/go-grpc v0.0.0-20210225150812-73cb765af46e
github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58
github.com/petar/GoLLRB v0.0.0-20210522233825-ae3b015fd3e9
github.com/rwcarlsen/goexif v0.0.0-20190401172101-9e8deecbddbd
github.com/shurcooL/component v0.0.0-20170202220835-f88ec8f54cc4
github.com/shurcooL/events v0.0.0-20181021180414-410e4ca65f48
github.com/shurcooL/github_flavored_markdown v0.0.0-20181002035957-2122de532470
github.com/shurcooL/go v0.0.0-20200502201357-93f07166e636
github.com/shurcooL/go-goon v0.0.0-20170922171312-37c2f522c041
github.com/shurcooL/gofontwoff v0.0.0-20180329035133-29b52fc0a18d
github.com/shurcooL/gopherjslib v0.0.0-20160914041154-feb6d3990c2c
github.com/shurcooL/highlight_diff v0.0.0-20170515013008-09bb4053de1b
github.com/shurcooL/highlight_go v0.0.0-20181028180052-98c3abbbae20
github.com/shurcooL/home v0.0.0-20181020052607-80b7ffcb30f9
github.com/shurcooL/htmlg v0.0.0-20170918183704-d01228ac9e50
github.com/shurcooL/httperror v0.0.0-20170206035902-86b7830d14cc
github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749
github.com/shurcooL/httpgzip v0.0.0-20180522190206-b1c53ac65af9
github.com/shurcooL/issues v0.0.0-20181008053335-6292fdc1e191
github.com/shurcooL/issuesapp v0.0.0-20180602232740-048589ce2241
github.com/shurcooL/notifications v0.0.0-20181007000457-627ab5aea122
github.com/shurcooL/octicon v0.0.0-20181028054416-fa4f57f9efb2
github.com/shurcooL/reactions v0.0.0-20181006231557-f2e0b4ca5b82
github.com/shurcooL/users v0.0.0-20180125191416-49c67e49c537
github.com/shurcooL/vfsgen v0.0.0-20200824052919-0d455de96546
github.com/shurcooL/webdavfs v0.0.0-20170829043945-18c3829fa133
github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d
github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e
github.com/spacemonkeygo/openssl v0.0.0-20181017203307-c2dcc5cca94a
github.com/spacemonkeygo/spacelog v0.0.0-20180420211403-2296661a0572
github.com/stvp/go-udp-testing v0.0.0-20201019212854-469649b16807
github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7
github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07
github.com/ucarion/urlpath v0.0.0-20200424170820-7ccc79b76bbb
github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8
github.com/warpfork/go-wish v0.0.0-20220906213052-39a1cc7a02d0
github.com/weaveworks/common v0.0.0-20230531151736-e2613bee6b73
github.com/whyrusleeping/base32 v0.0.0-20170828182744-c30ac30633cc
github.com/whyrusleeping/bencher v0.0.0-20190829221104-bb6607aa8bba
github.com/whyrusleeping/cbor v0.0.0-20171005072247-63513f603b11
github.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f
github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1
github.com/whyrusleeping/go-logging v0.0.0-20170515211332-0457bb6b88fc
github.com/whyrusleeping/go-notifier v0.0.0-20170827234753-097c5d47330f
github.com/whyrusleeping/mdns v0.0.0-20180901202407-ef14215e6b30
github.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415
github.com/xorcare/golden v0.6.1-0.20191112154924-b87f686d7542
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77
github.com/xrash/smetrics v0.0.0-20240312152122-5f08fbb34913
github.com/yugabyte/pgx/v5 v5.5.3-yb-2
gitlab.com/yawning/secp256k1-voi v0.0.0-20230925100816-f2616030848b
gitlab.com/yawning/tuplehash v0.0.0-20230713102510-df83abbf9a02
go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee
go4.org v0.0.0-20230225012048-214862532bf5
golang.org/x/build v0.0.0-20190111050920-041ab4dc3f9d
golang.org/x/exp v0.0.0-20240904232852-e7e105dedf7e
golang.org/x/lint v0.0.0-20200302205851-738671d3881b
golang.org/x/mobile v0.0.0-20190719004257-d2bd2a29d028
golang.org/x/perf v0.0.0-20180704124530-6e6d33e29852
golang.org/x/telemetry v0.0.0-20240521205824-bda55230c457
golang.org/x/xerrors v0.0.0-20240716161551-93cc26a95ae9
google.golang.org/genproto v0.0.0-20230530153820-e85fd2cbaebc
google.golang.org/genproto/googleapis/api v0.0.0-20240515191416-fc5f0ca64291
google.golang.org/genproto/googleapis/rpc v0.0.0-20240515191416-fc5f0ca64291
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7
grpc.go4.org v0.0.0-20170609214715-11d0a25b4919
honnef.co/go/tools v0.0.1-2020.1.4
howett.net/plist v0.0.0-20181124034731-591f970eefbb
sourcegraph.com/sqs/pbtypes v0.0.0-20180604144634-d3ebe8f20ae4

[BigLep]

In light of #12467, I elaborated in the issue description. I did this because this an action item I think maintainers should take on as a preventative measure.

@Stebalien's original issue description:

(lotus only)
Given that so many users build directly from lotus master, we should add a CI check to make sure that lotus master always depends on released modules (where possible). This is can be done by checking for dependencies on commits in go list -m all. This check would only apply to PRs against master.

[Stebalien]

153 out of 672 dependencies use non-releaesd versions

Note: The idea was to require released versions for dependencies with released versions. v0.0.0 versions don't have any releases and would be exempt. We only have 5 such packages:

github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7
github.com/xorcare/golden v0.6.1-0.20191112154924-b87f686d7542
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7

And two pre-release packages:

github.com/filecoin-project/go-state-types v0.15.0-dev
github.com/filecoin-project/go-data-transfer/v2 v2.0.0-rc7

[BigLep]

Proposed IPDX SoW that we've been discussing (thanks @galargh):

The CI workflow:

• if a skip check label/title exists, clear any existing comments and exit
• check whether all* dependencies** are released
• if so, update the comment and exit
• otherwise, comment on the PR with a warning and fail the check

• excluding known exceptions (v0.0.0 for example)
  ** including transitive

Complexity: testing

@galargh: one point of clarification: this will only flag on new or changed dependencies (regardless if they are direct or transitive). Basically I want to make sure that we don’t encumber future PRs with “sins of the past”. I just want to make sure we hold the line and don’t repeat the sins of the past, but it's a separate effort to clean up past sins.

[Stebalien]

one point of clarification

One option is to have a separate config file (json) for listing all the "excepted" dependencies.