Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go/cli: Coalesce identical channels by default #985

Conversation

achim-k
Copy link
Contributor

@achim-k achim-k commented Oct 2, 2023

Public-Facing Changes

go/cli: Coalesce identical channels by default

Description

Changes the default behavior of mcap merge to automatically coalesce identical channels. Channels are considered identical when their topic, schema, message encoding and metadata equals. One can force coalescing of channels with different metadata with --coalesce-channels force. Channel coalescing can also be completely disabled with --coalesce-channels none.

Copy link
Member

@jtbandes jtbandes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 The idea seems fine to me but I don't think I'm the best person to review the code changes.

@defunctzombie
Copy link
Contributor

@wkalt @james-rms thoughts?

@achim-k
Copy link
Contributor Author

achim-k commented Oct 30, 2023

Friendly ping @wkalt / @james-rms / @narasaka

go/cli/mcap/cmd/merge.go Outdated Show resolved Hide resolved
}

hash := hasher.Sum(nil)
return hex.EncodeToString(hash)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems odd to me that we store the hash as a string rather than a fixed size value - won't block the PR, just seems odd.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like channelIdByHash should key off the result of hasher.Sum().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Should I do the same for the schemaIDByHash map ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that would be good.

Changes the default behavior of `mcap merge` to automatically coalesce
identical channels. Channels are considered identical when their topic,
schema, message encoding and metadata equals. One can force coalescing
of channels with different metadata with `--coalesce-channels force`.
Channel coalescing can also be completely disabled with
`--coalesce-channels none`.
@achim-k achim-k force-pushed the achim/fg-4945-mcap-option-for-merge-to-attempt-to-coalesce-channelsschemas branch from 6fce3dc to eb8491f Compare November 2, 2023 19:44
@github-actions github-actions bot requested a deployment to mcap (Preview) November 2, 2023 22:53 Abandoned
@github-actions github-actions bot requested a deployment to mcap (Preview) November 2, 2023 23:08 Abandoned
@achim-k achim-k merged commit 3a38aea into main Nov 2, 2023
24 of 25 checks passed
@achim-k achim-k deleted the achim/fg-4945-mcap-option-for-merge-to-attempt-to-coalesce-channelsschemas branch November 2, 2023 23:10
pezy pushed a commit to pezy/mcap that referenced this pull request Jan 11, 2024
### Public-Facing Changes
go/cli: Coalesce identical channels by default

### Description
Changes the default behavior of `mcap merge` to automatically coalesce
identical channels. Channels are considered identical when their topic,
schema, message encoding and metadata equals. One can force coalescing
of channels with different metadata with `--coalesce-channels force`.
Channel coalescing can also be completely disabled with
`--coalesce-channels none`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants