-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
scylla_node: add ScyllaNode.dump_sstables() #484
Conversation
5b6bf04
to
4cd1e47
Compare
with the proposed change, we can have something like diff --git a/compaction_test.py b/compaction_test.py
index 4b4059dd..53100bca 100644
--- a/compaction_test.py
+++ b/compaction_test.py
@@ -180,19 +180,10 @@ class TestCompaction(Tester):
node.flush()
node.compact()
- sstable_dumps = node.run_scylla_sstable('dump-data', ['--merge'],
- keyspace=self.KEYSPACE_NAME)
- numfound = 0
- jsoninfo = []
- for stdout, _ in sstable_dumps.values():
- partitions = json.loads(stdout)['sstables']['anonymous']
- jsoninfo += partitions
- for partition in partitions:
- if 'tombstone' in partition:
- numfound += 1
-
+ partitions = node.dump_sstables(keyspace=self.KEYSPACE_NAME)
+ numfound = sum('tombstone' in partition for partition in partitions)
logger.debug(f'Number of tombstones found on node {node.name}: {numfound}')
- return numfound, jsoninfo
+ return numfound, partitions
def _test_compaction_delete_tombstone_gc(self, tombstone_gc_mode='repair', node_num: int = 2,
r_factor: int = None, delete_keys: bool = True, partition_num: int = 100): after https://github.com/scylladb/scylla-dtest/pull/3351 lands |
ccmlib/scylla_node.py
Outdated
all_partitions: List[Dict[str, Any]] = [] | ||
for stdout, _ in sstable_dumps.values(): | ||
partitions = json.loads(stdout)['sstables']['anonymous'] | ||
all_partitions += partitions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that this will possibly mash together partitions from different tables. Which might be confusing. I think we should enforce that the user always passes a keyspace and exactly one table parameter. I think this is how most users will use this anyway. The rest (if any) can call run_scylla_sstable()
instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed accordingly.
e183ab2
to
c9187e8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The tagline of comment of the method is outdated, LGTM otherwise.
v2
|
c9187e8
to
bfd235d
Compare
v3:
|
`ScyllaNode.dump_sstables()` is a wrapper around `ScyllaNode.run_scylla_sstable()`. it provides a more user friendly interface than the latter. it is introduced so that tests can use it with less pain when migrating from `node.run_sstable2json()` to `node.run_scylla_sstable()`. Signed-off-by: Kefu Chai <[email protected]>
bfd235d
to
2fe3d89
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
ScyllaNode.dump_sstables()
is a wrapper aroundScyllaNode.run_scylla_sstable()
. it provides a more user friendly interface than the latter. it is introduced so that tests can use it with less pain when migrating fromnode.run_sstable2json()
tonode.run_scylla_sstable()
.