Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

help running a specific test #201

Open
SgtPooki opened this issue Mar 30, 2024 · 3 comments
Open

help running a specific test #201

SgtPooki opened this issue Mar 30, 2024 · 3 comments

Comments

@SgtPooki
Copy link
Member

SgtPooki commented Mar 30, 2024

I'm really confused still. i'm not a gopher, and I have very little familiarity with go testing

https://github.com/ipfs/gateway-conformance/blob/main/docs/examples.md doesn't help.

I'm trying to debug a test: TestCors/GET_Responses_from_Gateway_should_include_CORS_headers_allowing_JS_from_other_origins_to_read_the_data_cross-origin

I want to run that test, and only that test but I have no idea how to do that.

I've seen where the specs are defined,

var (
TrustlessGatewayRaw = Leaf{"trustless-block-gateway", stable}
TrustlessGatewayCAR = Leaf{"trustless-car-gateway", stable}
TrustlessGatewayCAROptional = Leaf{"trustless-car-gateway-optional", stable}
TrustlessGatewayIPNS = Leaf{"trustless-ipns-gateway", stable}
TrustlessGateway = Collection{"trustless-gateway", []Spec{TrustlessGatewayRaw, TrustlessGatewayCAR, TrustlessGatewayCAROptional, TrustlessGatewayIPNS}}
PathGatewayUnixFS = Leaf{"path-unixfs-gateway", stable}
PathGatewayIPNS = Leaf{"path-ipns-gateway", stable}
PathGatewayTAR = Leaf{"path-tar-gateway", stable}
PathGatewayDAG = Leaf{"path-dag-gateway", stable}
PathGatewayRaw = Leaf{"path-raw-gateway", stable}
PathGateway = Collection{"path-gateway", []Spec{PathGatewayUnixFS, PathGatewayIPNS, PathGatewayTAR, PathGatewayDAG, PathGatewayRaw}}
SubdomainGatewayIPFS = Leaf{"subdomain-ipfs-gateway", stable}
SubdomainGatewayIPNS = Leaf{"subdomain-ipns-gateway", stable}
SubdomainGateway = Collection{"subdomain-gateway", []Spec{SubdomainGatewayIPFS, SubdomainGatewayIPNS}}
DNSLinkGateway = Leaf{"dnslink-gateway", stable}
RedirectsFile = Leaf{"redirects-file", stable}
)
, but CORS isn't listed in that file anywhere..

where do I see a list of all the specs I can ignore, or tests I can skip?

@SgtPooki
Copy link
Member Author

SgtPooki commented May 1, 2024

the go test skip and run flags are NOT actual regex, and they don't make consistent sense. At least not in the context of gateway-conformance.

trying to skip TestDNSLinkGatewayUnixFSDirectoryListing/Backlink_on_root_CID_should_be_hidden_%28TODO:_cleanup_Kubo-specifics%29_%28direct_HTTP%29/Body with -skip ".*TODO.*" should work, but it doesnt.

I'm not sure what's happening here, but this is basically a blocker for making decent progress on ipfs/helia-verified-fetch#67

Also, #186 and other test failures due to some specific header formatting checking also makes unblocking ipfs/helia-verified-fetch#67 a pain.

@SgtPooki
Copy link
Member Author

SgtPooki commented May 1, 2024

Note that i installed gateway-conformance binary directly to ensure that execa & docker weren't fiddling with things, but $HOME/go/bin/gateway-conformance test --gateway-url=http://localhost:3441 --subdomain-url=http://localhost:3442 --verbose --json gwc-report-direct.json -- -skip '.*TODO.*' still fails to skip tests as desired.

@SgtPooki
Copy link
Member Author

SgtPooki commented May 1, 2024

it seems like I can skip the TODO tests with -skip '.*/.*TODO.*'

Control case

$HOME/go/bin/gateway-conformance test --gateway-url=http://localhost:3441 --subdomain-url=http://localhost:3442 --verbose --json gwc-report-direct.json
cat gwc-report-direct.json| grep 'TODO' | wc -l # 540

With skip flag

$HOME/go/bin/gateway-conformance test --gateway-url=http://localhost:3441 --subdomain-url=http://localhost:3442 --verbose --json gwc-report-direct.json -- -skip '.*/.*TODO.*'
cat gwc-report-direct.json| grep 'TODO' | wc -l # 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant