chore: easy-to-run docker compose setup for curio+forest #4848
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,103 @@ | ||||||
| ##################################### | ||||||
| ARG LOTUS_IMAGE | ||||||
| # hadolint ignore=DL3006 | ||||||
| FROM ${LOTUS_IMAGE} as lotus-test | ||||||
| FROM golang:1.22.3-bullseye AS curio-builder | ||||||
|
There was a problem hiding this comment. How about using There was a problem hiding this comment. I'd like to keep it as similar to the "official" curio build environment as possible. There was a problem hiding this comment.
Suggested change
There was a problem hiding this comment. The curio team uses |
||||||
|
|
||||||
| RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates build-essential clang ocl-icd-opencl-dev ocl-icd-libopencl1 jq libhwloc-dev | ||||||
|
|
||||||
| ENV XDG_CACHE_HOME="/tmp" | ||||||
|
|
||||||
| ### taken from https://github.com/rust-lang/docker-rust/blob/master/1.63.0/buster/Dockerfile | ||||||
| ENV RUSTUP_HOME=/usr/local/rustup \ | ||||||
| CARGO_HOME=/usr/local/cargo \ | ||||||
| PATH=/usr/local/cargo/bin:$PATH \ | ||||||
| RUST_VERSION=1.63.0 | ||||||
|
There was a problem hiding this comment.
Suggested change
There was a problem hiding this comment. Good catch. Updated. |
||||||
|
|
||||||
| SHELL ["/bin/bash", "-o", "pipefail", "-c"] | ||||||
|
|
||||||
| RUN set -eux; \ | ||||||
| dpkgArch="$(dpkg --print-architecture)"; \ | ||||||
| case "${dpkgArch##*-}" in \ | ||||||
| amd64) rustArch='x86_64-unknown-linux-gnu'; rustupSha256='5cc9ffd1026e82e7fb2eec2121ad71f4b0f044e88bca39207b3f6b769aaa799c' ;; \ | ||||||
| arm64) rustArch='aarch64-unknown-linux-gnu'; rustupSha256='e189948e396d47254103a49c987e7fb0e5dd8e34b200aa4481ecc4b8e41fb929' ;; \ | ||||||
| *) echo >&2 "unsupported architecture: ${dpkgArch}"; exit 1 ;; \ | ||||||
| esac; \ | ||||||
| url="https://static.rust-lang.org/rustup/archive/1.25.1/${rustArch}/rustup-init"; \ | ||||||
| wget --quiet "$url"; \ | ||||||
| echo "${rustupSha256} *rustup-init" | sha256sum -c -; \ | ||||||
| chmod +x rustup-init; \ | ||||||
| ./rustup-init -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION --default-host ${rustArch}; \ | ||||||
| rm rustup-init; \ | ||||||
| chmod -R a+w $RUSTUP_HOME $CARGO_HOME; \ | ||||||
| rustup --version; \ | ||||||
| cargo --version; \ | ||||||
| rustc --version; | ||||||
|
|
||||||
| RUN git clone --depth 1 --branch v1.23.1 https://github.com/filecoin-project/curio.git /opt/curio | ||||||
|
|
||||||
| WORKDIR /opt/curio | ||||||
|
|
||||||
| ### make configurable filecoin-ffi build | ||||||
| ARG FFI_BUILD_FROM_SOURCE=0 | ||||||
| ENV FFI_BUILD_FROM_SOURCE=${FFI_BUILD_FROM_SOURCE} | ||||||
|
|
||||||
| RUN make clean deps | ||||||
|
|
||||||
| ARG RUSTFLAGS="" | ||||||
| ARG GOFLAGS="" | ||||||
|
|
||||||
| RUN make build | ||||||
|
|
||||||
| ##################################### | ||||||
| FROM ubuntu:22.04 AS curio-all-in-one | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
|
|
||||||
| RUN apt-get update && \ | ||||||
| apt-get install -y --no-install-recommends dnsutils vim curl aria2 && \ | ||||||
| apt-get clean && \ | ||||||
| rm -rf /var/lib/apt/lists/* | ||||||
|
|
||||||
| # Copy libraries and binaries from curio-builder | ||||||
| COPY --from=curio-builder /etc/ssl/certs /etc/ssl/certs | ||||||
| COPY --from=curio-builder /lib/*/libdl.so.2 /lib/ | ||||||
| COPY --from=curio-builder /lib/*/librt.so.1 /lib/ | ||||||
| COPY --from=curio-builder /lib/*/libgcc_s.so.1 /lib/ | ||||||
| COPY --from=curio-builder /lib/*/libutil.so.1 /lib/ | ||||||
| COPY --from=curio-builder /usr/lib/*/libltdl.so.7 /lib/ | ||||||
| COPY --from=curio-builder /usr/lib/*/libnuma.so.1 /lib/ | ||||||
| COPY --from=curio-builder /usr/lib/*/libhwloc.so.* /lib/ | ||||||
| COPY --from=curio-builder /usr/lib/*/libOpenCL.so.1 /lib/ | ||||||
|
|
||||||
| # Setup user and OpenCL configuration | ||||||
| RUN useradd -r -u 532 -U fc && \ | ||||||
| mkdir -p /etc/OpenCL/vendors && \ | ||||||
| echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd | ||||||
|
|
||||||
| # Environment setup | ||||||
| ENV FILECOIN_PARAMETER_CACHE=/var/tmp/filecoin-proof-parameters \ | ||||||
| LOTUS_MINER_PATH=/var/lib/lotus-miner \ | ||||||
| LOTUS_PATH=/var/lib/lotus \ | ||||||
| CURIO_REPO_PATH=/var/lib/curio | ||||||
|
|
||||||
| # Copy binaries and scripts | ||||||
| COPY --from=lotus-test /usr/local/bin/lotus /usr/local/bin/ | ||||||
| COPY --from=lotus-test /usr/local/bin/lotus-seed /usr/local/bin/ | ||||||
| COPY --from=lotus-test /usr/local/bin/lotus-shed /usr/local/bin/ | ||||||
| COPY --from=lotus-test /usr/local/bin/lotus-miner /usr/local/bin/ | ||||||
| COPY --from=curio-builder /opt/curio/curio /usr/local/bin/ | ||||||
| COPY --from=curio-builder /opt/curio/sptool /usr/local/bin/ | ||||||
|
|
||||||
| # Set up directories and permissions | ||||||
| RUN mkdir /var/tmp/filecoin-proof-parameters \ | ||||||
| /var/lib/lotus \ | ||||||
| /var/lib/lotus-miner \ | ||||||
| /var/lib/curio && \ | ||||||
| chown fc: /var/tmp/filecoin-proof-parameters /var/lib/lotus /var/lib/lotus-miner /var/lib/curio | ||||||
|
|
||||||
| # Define volumes | ||||||
| VOLUME ["/var/tmp/filecoin-proof-parameters", "/var/lib/lotus", "/var/lib/lotus-miner", "/var/lib/curio"] | ||||||
|
|
||||||
| # Expose necessary ports | ||||||
| EXPOSE 1234 2345 12300 4701 32100 | ||||||
|
|
||||||
| CMD ["/bin/bash"] | ||||||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this built on every PR and subsequent commits? If so, it'd be better to save it, the same way we do with the Lotus Dockefiles.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll change it to build on demand.