Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[distGB]fix the problem when graph has few nodes or edges in distributed partition #7824

Merged
merged 20 commits into from
Oct 18, 2024

Conversation

CfromBU
Copy link
Collaborator

@CfromBU CfromBU commented Oct 14, 2024

this pr fix the issue related to #7793. When the number of nodes or edges is less than the number of partitions, partition can be created normally.

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 14, 2024

To trigger regression tests:

  • @dgl-bot run [instance-type] [which tests] [compare-with-branch];
    For example: @dgl-bot run g4dn.4xlarge all dmlc/master or @dgl-bot run c5.9xlarge kernel,api dmlc/master

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 14, 2024

Commit ID: 4efd6d40f22c07f8f5d9d4e7b753063b2ddda620

Build ID: 1

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 14, 2024

Commit ID: f63864cdaf4050310cb324dca1935cab44f930d1

Build ID: 2

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 14, 2024

Commit ID: c6a3096731f3f4e793761f6b17d0c502fcabd2bb

Build ID: 3

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 14, 2024

Commit ID: 156323d4700ca0de74d4f2a4a77b16c6628b3ddf

Build ID: 4

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

python/dgl/distributed/partition.py Outdated Show resolved Hide resolved
@@ -355,6 +356,33 @@ def _process_partition_gb(
return indptr, indices[sorted_idx], edge_ids[sorted_idx]


def update_node_map(node_map_val, end_ids_per_rank, id_ntypes, prev_last_id):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copied from partition.py or modified accordingly? please add a comment to clarify it

tools/distpartitioning/dataset_utils.py Outdated Show resolved Hide resolved
@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 14, 2024

Commit ID: f46d64bb1b50c2426d220585156096d9c15472b5

Build ID: 5

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 15, 2024

Commit ID: 7e2b3f76e5794fcd4351a8ced29fb85c3b7117ad

Build ID: 6

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 15, 2024

Commit ID: 7ed9d231cf251edb4386c6ac130d43f4fda38b46

Build ID: 7

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 15, 2024

Commit ID: 3b60ea39b49bdf735ffd4d39a9fcf75e098cac30

Build ID: 8

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 15, 2024

Commit ID: 2932a6b8090202c8118cbeeaa26eb5c990164439

Build ID: 9

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 15, 2024

Commit ID: 8dba4d464e0f8efdf333705bd3b372c22cd5e6f5

Build ID: 10

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 16, 2024

Commit ID: 5f0ccc3a93b19fb952ed2556b83597dc8e4d4fe8

Build ID: 11

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 16, 2024

Commit ID: 8789ce280ee6469be00b3d3593b22590c4715e03

Build ID: 12

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@dgl-bot
Copy link
Collaborator

dgl-bot commented Oct 17, 2024

Commit ID: 7cb05b992b94e5b4a92b1e8e4dcc28087b1e4a59

Build ID: 13

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

@Rhett-Ying Rhett-Ying merged commit 88f109f into dmlc:master Oct 18, 2024
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Dist] check whether dist partition pipeline works with few num_nodes/edges
3 participants