Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up build scripts for all ARM build artifacts and the firmware #27

Open
4 of 5 tasks
dl1com opened this issue Jul 7, 2020 · 3 comments · Fixed by #28
Open
4 of 5 tasks

Set up build scripts for all ARM build artifacts and the firmware #27

dl1com opened this issue Jul 7, 2020 · 3 comments · Fixed by #28
Labels
devops enhancement New feature or request

Comments

@dl1com
Copy link
Contributor

dl1com commented Jul 7, 2020

  • Add a folder build_scripts where buildscripts for the various artifacts reside.
    This enables re-usage of the scripts for the end-user as well as for the automated Github actions.

  • Add script create_sysroot.sh
    This script pulls the plutosdr-fw repo with a given Tag or Commit hash from the Analog Devices Github account and alters its content according to our needs (compilation of fftw, libconfig, libfec and liquid-dsp).
    The result is called custom sysroot. The sysroot gets zipped and uploaded to Github releases to be available end-users and the build_binaries.sh script (see below).

  • Add script build_standalone_binaries.sh
    This script also makes use of the custom sysroot artifact. As it would be to much effort to re-generate the custom sysroot on every build of the binaries (especially for Github actions), the chosen version of the custom sysroot artifact is downloaded from Github releases instead.

  • Add script build_firmware.sh
    This script makes use of the custom sysroot which was created with the script mentioned above.
    Additionally, it patches and configures the Linux Kernel according to our needs and configures the Linux userspace.
    Finally, it builds the firmware image called pluto.frm.

  • Update manual.hnap.de accordingly

  • No actions for the x86 binaries yet (they are already implemented as Github actions in .github/workflows)

image

@dl1com dl1com added enhancement New feature or request devops labels Jul 7, 2020
@lukasostendorf
Copy link
Collaborator

I've been working on a first version of this. Current state:

create_sysroot.sh

  • ARGS: needs path to the sysroot folder
  • downloads and installs all additional libraries and header files to the sysroot folder.
  • creates a .tar.gz of the finished sysroot folder in the directory where the script is located (build_scripts)

build_firmware.sh

  • ARGS: needs path to Xilinx Toolchain installation; path to sysroot folder.
  • if no plutosdr-fw folder is downloaded yet:
    • downloads the plutosdr-fw repo to build_scripts/plutosdr-fw
    • verifies that plutosdr-fw git tag matches the tag against which we want to build (v0.32)
    • applies RT kernel patch and makes changes to kernel and buildroot config
  • call create_sysroot.sh
  • build the standalone applications
  • build the firmware
  • copy the firmware image to the `build_scripts' folder

Open questions

I do not want to download the whole firmware repo from scratch every time. It takes much time to download and even more time to compile the linux kernel from scratch every time. What is a good way to check whether it is already downloaded? Currently plutosdr-fw download path is fixed and a check whether the dir exists is performed.

Do we want to let the user choose where to create the plutosdr-fw dir?

Kernel patching and changing the buildroot+kernel configuration must be only done once or it will fail. How to check if this has already been done? The current script does not properly handle this.

@dl1com
Copy link
Contributor Author

dl1com commented Jul 10, 2020

Please see my comments in your commit 05bd621

Open questions

I do not want to download the whole firmware repo from scratch every time. It takes much time to download and even more time to compile the linux kernel from scratch every time. What is a good way to check whether it is already downloaded? Currently plutosdr-fw download path is fixed and a check whether the dir exists is performed.

Do we want to let the user choose where to create the plutosdr-fw dir?

I'd propose a fixed directory inside a designated build directory within the repo.

Kernel patching and changing the buildroot+kernel configuration must be only done once or it will fail. How to check if this has already been done? The current script does not properly handle this.

I'd propose touching a flag file and check if it exists prior to applying the Kernel modifications. This method won't be sufficient for many cases (e.g. changing to a different Tag / Branch in plutosdr-fw), but for our current quasi-static use case (always v0.32), this would be working, I guess.

lukasostendorf added a commit that referenced this issue Jul 11, 2020
dl1com pushed a commit that referenced this issue Jul 12, 2020
* Add scripts to automate firmware and sysroot build process

Related to #27
@dl1com dl1com linked a pull request Jul 12, 2020 that will close this issue
@dl1com
Copy link
Contributor Author

dl1com commented Jul 16, 2020

While setting up CI for the build_standalone_binaries.sh script (develop...workflow), I became aware of something we might want to optimize.
Currently, the custom sysroot would be built each time we trigger a build, which increases build time unnecessarily to about 3 minutes.
We could reduce it by either caching the custom sysroot folder or downloading an already customized sysroot folder:

  • Caching the custom sysroot folder so it persists between builds
  • Downloading an already customized folder
    • This would be pretty similar to what was already intended for v1.0.0: we supply a tar with the custom sysroot e.g. in github releases and fetch it before running the build script for the binaries.
    • Though, it would be something like a circular dependency, as we are relying on something which has not been released yet. We would have to assign a proper tag/release name to the custom sysroot which is currently used for development first.
    • If we wanted to supply a custom sysroot to the users anyway, this would be a good solution. But as we have really good build scripts now, the users can easily build it on their own and I don't think it's necessary to supply it because of the users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants