From 70b0756b2ae90324ef00a46c5127f77c4f4ea100 Mon Sep 17 00:00:00 2001 From: "Mark C. Miller" Date: Thu, 3 Oct 2024 11:34:16 -0700 Subject: [PATCH 1/5] add event --- Events/2024-10-reproduciblehpc.md | 83 +++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 Events/2024-10-reproduciblehpc.md diff --git a/Events/2024-10-reproduciblehpc.md b/Events/2024-10-reproduciblehpc.md new file mode 100644 index 000000000..9b621ea21 --- /dev/null +++ b/Events/2024-10-reproduciblehpc.md @@ -0,0 +1,83 @@ +# Community Workshop on Practical Reproducibility in HPC - Registration and Call for Presentations + + +Understanding unique challenges and practical solutions of reproducibility in High-Performance Computing (HPC). + + +- Deadline for Call for Presentations: Oct. 18, 2024 +- Deadline for Early Bird Registration: Oct. 25, 2024 +- Event Date: November 18, 2024 +- Location: Atlanta, GA +- Event Website: https://reproduciblehpc.org +- Organizers: Kate Keahey and Marc Richardson + +We are excited to announce call for presentations/abstracts for the upcoming Workshop on Practical Reproducibility in HPC, to be held on November 18, 2024, in Atlanta, GA. +This workshop aims to address the unique challenges of reproducibility in High-Performance Computing (HPC) and advance practical solutions for those challenges. Sign up or apply to present at [reproduciblehpc.org](reproduciblehpc.org). + +Event Information | Details +:--- | :--- +Event Name | [Community Workshop on Practical Reproducibility in HPC](https://reproduciblehpc.org) +Event Date| Nov. 18, 2024 +Website | https://reproduciblehpc.org +Call for Presentations Deadline | Oct. 18, 2024 +Early Bird Registration Deadline | Oct. 25, 2024 +Registration and Other Information | https://www.eventbrite.com/e/community-workshop-on-practical-reproducibility-in-hpc-tickets-1000658535617 + +### Description + +This workshop -- co-located with a major Computer Science conference taking place in Atlanta, GA on November 18, 2024 -- aims to address the unique challenges of reproducibility in High-Performance Computing (HPC) and advance practical solutions for those challenges. + +There is a broad agreement that in the digital age science should be shared digitally -- through artifacts such as code and data -- and that adopting practices enabling or facilitating reproducibility of computational results can lead to more robust science and increased scientific productivity. Many conferences encourage this practice through artifact evaluation initiatives -- yet, the tools, services, and practices that support both authors and reviewers in sharing the research are still often either insufficient or lacking. This is particularly true in high performance computing (HPC) where packaging experiments for reproducibility presents unique challenges given their need for deep reconfiguration; unique and cutting edge hardware; and significant scales. + +The objective of this workshop is to bring together experiment authors and reviewers with experience or insight into packaging and reviewing computer science experiments on open platforms to discuss challenges, share ideas and tips, and formulate a path forward on how to best support reproducibility for computer science research. Since many such initiatives were supported by the Chameleon platform ([www.chameleoncloud.org](www.chameleoncloud.org)), this workshop will also double as a Chameleon User Meeting with the objective to bring together both existing, potential, and future users of the platform. + +**As in previous Chameleon User Meetings, the organizers will reimburse travel expenses of up to $1,500 for the presenting authors of the top 10 selected presentation abstracts (one author per abstract). Please, take a look at the Call For Presentations below for details.** + +### Call for Presentation Proposal Guidelines + +**Presentation proposals should be in PDF format (2 pages maximum), submitted (by Oct. 18, 2024) to presentations@chameleoncloud.org, and should include the following information:** + +**Preamble**: + +- Presentation title +- Presenter name(s) and affiliation(s) +- Reproducibility platforms/tools used or projected (e.g., Chameleon, other testbed, containers, notebooks) + +**Proposal Body Sections**: + +- *Project Description (one paragraph or less)*: + A brief but clear description of the Computer Science experiment you are packaging or reproducing: What research problem does it address? (What is the hypothesis? What is the challenge/trade-off? Why is it significant?) How does the experiment capture this research problem? +- *Reproducibility Approach (one paragraph or less)*: + What is the description of the experiment you are either packaging or reproducing? What resources does it need and how many? How do those resources need to be configured? What is the experimental environments in which the experiment should run? How are they configured? What does the experiment body consist of? How long and in what manner is the experiment run? What types of data does it produce and how much? How is the data analyzed? What platforms or tools did you use in your experiment? +- *Lessons Learned*: + Good and bad. What were the most important obstacles to either packaging for reproducibility or reproducing the experiment you encountered – and how did you overcome them? Were they on the infrastructure level (hardware availability), configuration level (how the hardware is configured, i.e., “creating an MPI cluster”), executing the experimental workflow, managing or analyzing data? What strategies were effective? How did you decide (or what guidelines did you give to reviewers) as to when the experiment is reproduced? What suggestions or wishes do you have on how support for reproducibility should improve? Specifically, what should improve in (1) the ability to set up an experimental environment, (2) the ability to manage the experiment body, and (3) data analysis? Which of those desired capabilities are the most important ones? Any procedural insights on how to organize reproducibility initiatives? +- *Impact and Future Directions (one paragraph or less)*: + How has prioritizing reproducibility influenced your HPC research or applications? How can the HPC community better integrate reproducibility into mainstream research and education? + +**Selection Criteria** + +Presentations will be selected based on their relevance to HPC-specific reproducibility challenges, potential for fostering discussion, and the insights they offer to the community. We particularly encourage submissions that: +- Have an insightful, detailed, and well communicated “lessons learned” section +- Have a good discussion/analysis of specific reproducibility challenges in HPC +- Share experiences from HPC-focused reproducibility initiatives + +**Travel Support** + +For the top 10 selected abstracts, we will reimburse travel expenses of up to $1,500 for the presenting authors (one per abstract). Submitting the presentation proposal doubles as travel support application. + +**Workshop Outcomes** + +The workshop aims to produce a report capturing the community's collective knowledge and recommendations for advancing practical reproducibility in HPC. Your presentations and participation will directly contribute to this valuable resource for the HPC community. + +Please send your submissions to presentations@chameleoncloud.org. Visit our workshop website ([reproduciblehpc.org](reproduciblehpc.org)) for more information. For any questions, contact us at contact@chameleoncloud.org or via the Chameleon users list. + +We look forward to your contributions and to seeing you in Atlanta this November! + +#### Publication date: October 3, 2024 + +#### Contributed by: [Marc Richardson](https://www.linkedin.com/in/marctrichardson/) + + From e2a756cb0cc8a7935d0ee300cae0685e4cad1668 Mon Sep 17 00:00:00 2001 From: "Mark C. Miller" Date: Thu, 3 Oct 2024 11:43:14 -0700 Subject: [PATCH 2/5] fix broken links --- Events/2024-10-reproduciblehpc.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Events/2024-10-reproduciblehpc.md b/Events/2024-10-reproduciblehpc.md index 9b621ea21..ad2a3d752 100644 --- a/Events/2024-10-reproduciblehpc.md +++ b/Events/2024-10-reproduciblehpc.md @@ -12,7 +12,7 @@ Understanding unique challenges and practical solutions of reproducibility in Hi - Organizers: Kate Keahey and Marc Richardson We are excited to announce call for presentations/abstracts for the upcoming Workshop on Practical Reproducibility in HPC, to be held on November 18, 2024, in Atlanta, GA. -This workshop aims to address the unique challenges of reproducibility in High-Performance Computing (HPC) and advance practical solutions for those challenges. Sign up or apply to present at [reproduciblehpc.org](reproduciblehpc.org). +This workshop aims to address the unique challenges of reproducibility in High-Performance Computing (HPC) and advance practical solutions for those challenges. Sign up or apply to present at [reproduciblehpc.org](https://reproduciblehpc.org). Event Information | Details :--- | :--- @@ -69,7 +69,7 @@ For the top 10 selected abstracts, we will reimburse travel expenses of up to $1 The workshop aims to produce a report capturing the community's collective knowledge and recommendations for advancing practical reproducibility in HPC. Your presentations and participation will directly contribute to this valuable resource for the HPC community. -Please send your submissions to presentations@chameleoncloud.org. Visit our workshop website ([reproduciblehpc.org](reproduciblehpc.org)) for more information. For any questions, contact us at contact@chameleoncloud.org or via the Chameleon users list. +Please send your submissions to presentations@chameleoncloud.org. Visit our workshop website ([reproduciblehpc.org](https://reproduciblehpc.org)) for more information. For any questions, contact us at contact@chameleoncloud.org or via the Chameleon users list. We look forward to your contributions and to seeing you in Atlanta this November! From 1763375be94d82463131fc2c5d59ee5978e0720c Mon Sep 17 00:00:00 2001 From: "Mark C. Miller" Date: Thu, 3 Oct 2024 11:48:33 -0700 Subject: [PATCH 3/5] fix another link --- Events/2024-10-reproduciblehpc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Events/2024-10-reproduciblehpc.md b/Events/2024-10-reproduciblehpc.md index ad2a3d752..0943690f3 100644 --- a/Events/2024-10-reproduciblehpc.md +++ b/Events/2024-10-reproduciblehpc.md @@ -29,7 +29,7 @@ This workshop -- co-located with a major Computer Science conference taking plac There is a broad agreement that in the digital age science should be shared digitally -- through artifacts such as code and data -- and that adopting practices enabling or facilitating reproducibility of computational results can lead to more robust science and increased scientific productivity. Many conferences encourage this practice through artifact evaluation initiatives -- yet, the tools, services, and practices that support both authors and reviewers in sharing the research are still often either insufficient or lacking. This is particularly true in high performance computing (HPC) where packaging experiments for reproducibility presents unique challenges given their need for deep reconfiguration; unique and cutting edge hardware; and significant scales. -The objective of this workshop is to bring together experiment authors and reviewers with experience or insight into packaging and reviewing computer science experiments on open platforms to discuss challenges, share ideas and tips, and formulate a path forward on how to best support reproducibility for computer science research. Since many such initiatives were supported by the Chameleon platform ([www.chameleoncloud.org](www.chameleoncloud.org)), this workshop will also double as a Chameleon User Meeting with the objective to bring together both existing, potential, and future users of the platform. +The objective of this workshop is to bring together experiment authors and reviewers with experience or insight into packaging and reviewing computer science experiments on open platforms to discuss challenges, share ideas and tips, and formulate a path forward on how to best support reproducibility for computer science research. Since many such initiatives were supported by the Chameleon platform ([www.chameleoncloud.org](https://www.chameleoncloud.org)), this workshop will also double as a Chameleon User Meeting with the objective to bring together both existing, potential, and future users of the platform. **As in previous Chameleon User Meetings, the organizers will reimburse travel expenses of up to $1,500 for the presenting authors of the top 10 selected presentation abstracts (one author per abstract). Please, take a look at the Call For Presentations below for details.** From 87c6ed1c439c16c994a1ea50a60f42ed492d817c Mon Sep 17 00:00:00 2001 From: "Mark C. Miller" Date: Thu, 3 Oct 2024 12:12:11 -0700 Subject: [PATCH 4/5] fix topics --- Events/2024-10-reproduciblehpc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Events/2024-10-reproduciblehpc.md b/Events/2024-10-reproduciblehpc.md index 0943690f3..d3ac3fdbc 100644 --- a/Events/2024-10-reproduciblehpc.md +++ b/Events/2024-10-reproduciblehpc.md @@ -79,5 +79,5 @@ We look forward to your contributions and to seeing you in Atlanta this November From 6b0095f568411b3526bf0c73d0f182242318d958 Mon Sep 17 00:00:00 2001 From: Lois Curfman McInnes Date: Thu, 3 Oct 2024 19:23:44 -0400 Subject: [PATCH 5/5] Update 2024-10-reproduciblehpc.md small fix --- Events/2024-10-reproduciblehpc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Events/2024-10-reproduciblehpc.md b/Events/2024-10-reproduciblehpc.md index d3ac3fdbc..0bebe3dba 100644 --- a/Events/2024-10-reproduciblehpc.md +++ b/Events/2024-10-reproduciblehpc.md @@ -11,7 +11,7 @@ Understanding unique challenges and practical solutions of reproducibility in Hi - Event Website: https://reproduciblehpc.org - Organizers: Kate Keahey and Marc Richardson -We are excited to announce call for presentations/abstracts for the upcoming Workshop on Practical Reproducibility in HPC, to be held on November 18, 2024, in Atlanta, GA. +We are excited to announce our call for presentations/abstracts for the upcoming Workshop on Practical Reproducibility in HPC, to be held on November 18, 2024, in Atlanta, GA. This workshop aims to address the unique challenges of reproducibility in High-Performance Computing (HPC) and advance practical solutions for those challenges. Sign up or apply to present at [reproduciblehpc.org](https://reproduciblehpc.org). Event Information | Details