Skip to content

Commit

Permalink
wrote article draft
Browse files Browse the repository at this point in the history
  • Loading branch information
gflofst committed Aug 22, 2024
1 parent 35c6426 commit 0e15af6
Showing 1 changed file with 45 additions and 0 deletions.
45 changes: 45 additions & 0 deletions Articles/Blog/ACM-REP-23-24.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# The new ACM REP conference


#### Contributed by [Jay Lofstead](https://github.com/gflofst "Jay Lofstead GitHub Profile")

#### Publication date: September 10, 2024

The ACM REP conference helps build better software by highlighting developments in reproducibility and replicability for computational science.

A few years ago, growing out of Carlos Maltzahn, his student Ivo Jimenez, and my work, we saw momentum towards a critical mass of people interested in reproducibility and replicability challenges and successes for computational science. To help drive the community and grow those efforts, the topic of Ivo Jimenez's PhD thesis, we gathered prominent community members and started an Exploratory Interest Group (EIG) with the Association of Computing Machinery (ACM). Our goal was to explore how big this community was and see if we could make a catalyst and meeting point for these researchers and practitioners to help drive improving scientific software quality.

Within a few years, we felt like we had enough momentum and in Summer 2023, we had the first ACM REP conference. Even with short notice and not much advertising, we managed to get a critical mass of research papers and attendance that proved the community was both large enough and interested in participating.

For Summer 2023, we met in the Hay Barn at the University of California, Santa Cruz and online. With the success of that effort, we took the conference to Rennes, France in Summer 2024 to see what more convenient attendance for the European community would muster. We had slight more attendance and a different mix showing a fragmented community we could help bring together.

For 2025, we will bring ACM REP to Vancouver, Canada the last week of July examining what attendance may be like for something closer to east Asia, but not with the potential visa issues for something hosted in the USA.

At ACM REP 23, we had three keynotes each looking at a different aspect of sustainable software. Torsten Hoefler described his path into proper statistics for analyzing data and the impacts that had on quality. Juliana Feire presented on the challenges in many domains for data driven science. The last keynote was from Grigori Fursin laid out the efforts he has been involved with that trying to provide tools to support reproducible science. In all of these cases, the depth and breadth of available research topics were made clear and the challenges in achieving success are shown to be daunting. For us to achieve better scientific software, considerable research will be needed.

At ACM REP 24, our first keynote was from Anne-Laure Boulesteix talking about the challenges in machine learning and how reproducibility is particularly challenging as ML-infused approaches become an integral part of our computational science. Our second keynote was from Konrad Hinsen talking about the problems with platforms and how GUIX can give a portable, stable, reproducible base upon which to run reproducible computational science. While not a complete solution, it offers stronger infrastructure to make many of the ill-defined elements more about the science rather than the experimental environment.

At both years, we featured a wide variety of papers ranging from success stories to incremental, but difficult and important improvements to make reproducibility and replicability achievable. The discussions among attendees has also shown this to be a vibrant and collaborative group interested in advancing the field rather than trying to hide their own efforts.

Science is basically defined as something that can be reproduced in a new environment by someone else and achieve statistically the same results. Sometimes it is exactly and there are some other minor variations. Overall, that ability to recreate the results again is key for something to be considered **science**. With the importance of computational science to our scientific inquiry process. Ensuring it is science must be done or our results are little more than, "I got this result and I trust it. Therefore so should you."

This conference series was started hoping we could achieve a critical mass to foster community for sustainable computational science. The first two years have shown that this critical mass exists and is ready to grow.

Our latest effort is to encourage experience papers talking about lessons learned from reproducing other computational research. These experience papers can be from any conference or journal and will help justify the educational and research value in exploring existing work by offering a publication credit for writing up the results.

As with both previous years, we plan to continue offering tutorials with a stronger emphasis on hands-on tutorials rather than lecture style. This will help people learn how to use new tools.

If you wish to participate, please visit https://acm-rep.github.io/2025/ to keep abreast of the conference as it comes organizes.

Together, we can make better scientific software through better support for reproducibility and replicability.

### Author bio

Jay Lofstead is a Principal Member of Technical Staff at Sandia National Laboratories. His research interests focus around large scale data management and trusting scientific computing. In particular, he works on storage, IO, metadata, workflows, reproducibility, software engineering, machine learning, and operating system-level support for any of these topics. Broadly across these topics, he is also deeply interested in ethics related to these topics and computing in general and how to drive inclusivity across the computation-related science domains. Dr. Lofstead received his Ph.D. in Computer Science from the Georgia Institute of Technology in 2010.


<!---
Publish: No
Track: community
Topics: testing
--->

0 comments on commit 0e15af6

Please sign in to comment.