Start migrating to protobuf board configs #970
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Marked this as a draft because I still have some testing to do, and because I also first want to see how the board builds go in GitHub. |
|
Short summary of what this does, to encapsulate the details above:
This does not:
It does:
|
|
Future PR will include examples of what it looks like to whittle down the configs, as things are removed or replaced with protobuf representations, and will possibly/probably also include a tool or script or something to convert a whole .h to .json. |
|
Just a further note for anyone else reading this (including myself!), this is a step-us-in-the-right-direction PR change. This is what is leading us down the road of using .json as configs, and then the UF2 generation will be somewhat seamless. So our builds right now are this:
This will change us to:
Our build times will go down significantly and it will be much easier to build a custom config uf2 for non-technical users. Going to wait on other folks before I merge this one, but code looks good! |
|
This will be a 0.8.0 feature as we will completely remove legacy config at this point. |
|
Moving this to draft as we have a lot of things to do for protobuf |
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
from
a8b1989
to
b12dec5
Compare
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
from
b12dec5
to
f57b206
Compare
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
2 times, most recently
from
7c38545
to
a205bae
Compare
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
3 times, most recently
from
5a6e16b
to
60bffba
Compare
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
from
60bffba
to
090b91b
Compare
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
from
090b91b
to
c06d605
Compare
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
5 times, most recently
from
2c751a7
to
3542920
Compare
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
from
3542920
to
2454252
Compare
users should migrate to the previous release first, if they want to be supported without redoing their config
this also adds a sanity check for the now-expanded size of the storage sections in the flash
this is no longer relevant in the protobuf-as-board-config world
this lets a board config patched into the binary to specify values that are not in the user config, and have them be applied to the user config. essentially, on board boot, we now: 1. pb_decode in loadBoardConfig 2. this loads the defaults 3. this then populates the structure with values from the board config 4. pb_decode_ex in loadUserConfig 5. this does NOT load the defaults 6. this then splices values from the user config into the structure the BoardConfig.h files and initUnset... remain, but this will allow us to move off of that as we start getting tooling in place for the protobuf board configs
it was possible to never call pb_decode to initialize the config with defaults (e.g. if both the board and user configs were empty), this avoids that, in which case the calls are essentially identical
I think this is going to be a long road, but this is the kind of thing that's possible even without a board config in place
this starts the replacement of BoardConfig.h + migration code with shipped protobuf configs. this should help cleanup some code, and one day will hopefully replace the need for per-board builds entirely, greatly speeding up the GitHub build/creation of board binaries
also spit out the summary info just so people can see what happened happened
something about the protoc compiler (maybe just on Python 3.12?) doesn't like multiple -P path arguments, and until that's fixed, it's easier to just copy the files into one spot, which works fine
bsstephan
force-pushed
the
start-migrating-to-protobuf-board-configs
branch
from
2454252
to
441031a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a post-build CMake command (thanks @FeralAI!) which uses gp2040ce-binary-tools to write a Protobuf board config into the
.bin and.uf2 files generated by the normal build.A Protobuf board config is a second reserved space at the end of the flash, essentially the exact same idea as the user config, just in an adjacent block of space. It is used to replace the Protobuf defaults with specified values (for now, as a demo, just the boardVersion version number) from the build, in order to convey a specific configuration with less C++ code in
initUnset....For the moment this doesn't undo anything material in
initUnset..., but with this in place, the next step will be setting defaults in the config.proto file and overriding them with specific board config JSON files (when applicable) and removing the defines and C++ code accordingly. With time, then, all of the define-based config initialization code can go away, and we can stop doing per-board builds in GitHub. This doesn't interfere withinitUnset...until then, as whatever is not specified gets initialized normally.We won't need a .h file to understand board defaults anymore, because it'll be there, self-documented in the binary and viewable via gp2040ce-binary-tools. This should help diagnosis and maintenance, and hopefully we can get some vendors on board with this methodology (once all the porting is done) and make it easier for them to configure and ship properly-configured binaries without having to maintain code.
Worth noting that the board config is read by the firmware, but not modifyable, meaning the user config can be cleared to return to board defaults, same as before.
Again this is just step one of the process, but this is the step that lets us start strangling off the BoardConfig.h way of doing things.
NOTE: support for legacy (pre-protobuf) configs had to be removed because of competing expectations of what it meant for loading the config to "fail".