PINNSTRIPES (Physics-Informed Neural Network SurrogaTe for Rapidly Identifying Parameters in Energy Systems) 
conda create --name pinnstripes python=3.11conda activate pinnstripespip install -r requirements.txt
Located in pinn_spm_param
-
pinn_spm_param/preProcess: make the data from finite difference integration. Dobash exec.sh -
pinn_spm_param: the main script ismain.pywhich starts the training. Dobash exec_opt.shfor training in production mode. Dobash exec_noOpt.shfor training in debug mode. The debug mode is slower but allows for easily inspecting the tensor shapes. This will execute a training with no data. -
pinn_spm_param/postProcess: post-process the PINN training result. Link the correct model and log folder inexec.shand dobash exec.sh
Results may vary depending on your initial seed, but after step 3., pinn_spm_param/postProcess/Figures folder will contain (among other images) movies of the predicted solution. Left panel shows correlation between the PDE and the PINN solution, middle panel shows the time history of the predicted radial dependent Li concentration in each electrode as training advances, right panel shows the time history of the predicted electrolyte and cathode potentials as training advances.
Consider looking at the test suite in pinn_spm_param/tests, BayesianCalibration_spm/exec_test.sh, and .github/workflows/ci.yml to understand how to use the code
cd pinn_spm_param
bash convert_to_float32.sh will enable training in single precision mode.
bash convert_to_float64.sh will enable training in double precision mode.
cd pinn_spm_param
bash convert_to_simp.sh will enable training with simple electrochemical and transport properties
bash convert_to_nosimp.sh will enable training with realistic electrochemical and transport properties
Data contains experimental measurements of Uocp that are used to generate the Uocp functions in pinn_spm_param/util/generateOCP.py and pinn_spm_param/util/generateOCP_poly_mon.py
The training occurs in two stages. First, we use SGD training, and next LBFGS training (for refinement). The number of epochs for SGD training and LBFGS training can be controlled via EPOCHS and EPOCHS_LBFGS. The SGD occurs in batches but the LBFGS does one step per epoch. To avoid memory issues, LBFGS can be set to accumulate the gradient by batches and then do use the accumulated gradient once all the batches are processed. The number of batches for SGD is controlled with N_BATCH and for LBFGS with N_BATCH_LBFGS. A warm start period can be used for LBFGS via EPOCHS_START_LBFGS which typically prevents blowup after the first steps.
SGD can be deactivated by setting EPOCHS: 0 or by setting SGD: False in pinn_spm_param/input
LBFGS can be deactivated by setting EPOCHS_LBFGS: 0 or by setting LBFGS: False in pinn_spm_param/input
We use 4 different PINN losses
- Interior losses compute the residual of governing equations at interior collocation points. The number of collocation points by batch is controlled with
BATCH_SIZE_INT. - Boundary losses compute the residual of boundary conditions at boundary collocation points. The number of collocation points by batch is controlled with
BATCH_SIZE_BOUND. - Data losses evaluate the mismatch with provided data. The maximum batch size for data is controlled with
MAX_BATCH_SIZE_DATA. If this number is too low, the code will ignore some of the data. The amount of data ignored is printed at the beginning of training. - Regularization losses compute regularization conditions at regularization collocation points. The number of collocation points by batch is controlled with
BATCH_SIZE_REG. Some regularization loss require performing an integration. The number of points used for integrating over each domain is controlled byBATCH_SIZE_STRUCT. In the SPM case, no regularization is used.
The user may activate or deactivate each loss via the alpha parameter in main.py. The active or inactive losses are printed at the beginning of training.
The 4 PINN losses can be independently weighted via alpha : 1.0 1.0 0.0 0.0. In order, these coefficients weigh the interior loss, the boundary loss, the data loss, and the regularization loss.
Individual physics loss can be weighted via w_phie_int and others w_xxx_xxx
Learning rate is set with 2 parameters LEARNING_RATE_MODEL and LEARNING_RATE_MODEL_FINAL. The solver does half the epochs with the learning rate set with LEARNING_RATE_MODEL and then decays exponentially to LEARNING_RATE_MODEL_FINAL.
A similar workflow is set for the self-attention weights using LEARNING_RATE_WEIGHTS and LEARNING_RATE_WEIGHTS_FINAL.
The LBFGS learning rate can be set with LEARNING_RATE_LBFGS. The learning rate for LBFGS is dynamically adjusted during training to avoid instabilities. The learning rate given in the input file is a target that LBFGS tries to attain.
Initial conditions are strictly enforced. The rate at which we allow the neural net to deviate from the IC is given by HARD_IC_TIMESCALE.
To avoid exploding gradients from the Butler Volmer relation, we clip the interior of exponentials using EXP_LIMITER. Not used for SPM
An option to linearize the sinh to a linear function can be activated with LINEARIZE_J. This typically allows not having very large losses at the beginning of training.
The user can either demand a fixed set of collocation points carried through the training or demand that collocation points be random, i.e. randomly generated for each training step. Once LBFGS starts, the collocation points are held fixed.
Random collocation is incompatible with self-attention weights.
We can ask the PINN to gradually increase the time spanned by the collocation points via GRADUAL_TIME. If fixed collocation mode is used, then the collocation points' locations are gradually stretched over time only. The stretching schedule is controlled by RATIO_FIRST_TIME : 1. The stretching is done so that it reaches maximal time at the midpoint in the SGD Epoch.
Likewise, the time interval can gradually increase during the LBFGS training. If the user sets a warm start epoch number for LBFGS, the warm start is redone every time the time interval increases. The time interval stretching is controlled via N_GRADUAL_STEPS_LBFGS : 10 and GRADUAL_TIME_MODE_LBFGS: exponential. The time mode can be linear or exponential.
We can use self-attention weights with the flag DYNAMIC_ATTENTION_WEIGHTS. Each collocation point and data point is assigned a weight that is trained at each step of the SGD process. The user can decide when to start weight training by adjusting START_WEIGHT_TRAINING_EPOCH.
Weight annealing can be used by setting ANNEALING_WEIGHTS: True
Activation can be piloted via ACTIVATION. Available options are
tanhswishsigmoideluselugelu
Two architectures are available. Either a split architecture is used where each variable is determined by a separate branch. Otherwise, a merged architecture is available where a common latent space to all variables that depend of [t], [t,x], [t,r], or [t,x,r] is constructed. Choose architecture with MERGED flag. The number of hidden layers can be decided with hidden_unitsXXX flags in main.py.
Lastly, gradient pathology blocks, residual blocks, or fully connected blocks can be used.
The models can be used in hierarchical modes by setting HNN and HNNTIME. Examples are available in pinn_spm_param/tests. The hierarchy can be done by training models over the same spatio-temporal and parametric domain. It can be done by training lower hierarchy levels up until a threshold time. It can be done by training the lower hierarchy levels for a specific parameter set.
Under pinn_spm_param/integration_spm an implicit and an explicit integrator are provided to generate solutions of the SPM equations.
Under pinn_spm_param/integration_spm, run python main.py -nosimp -opt -lean to generate a rapid example of the SPM solution.
Under pinn_spm_param/preProcess, run python makeDataset_spm.py -nosimp -df ../integration_spm -frt 1 to generate a dataset usable by the PINN.
The implicit integration is recommended for fine spatial discretization due to the diffusive CFL constraint. For rapid integration using a coarse grid, the explicit integration will be preferable. The explicit integrator automatically adjusts the timestep based on the CFL constraint.
Under pinn_spm_param/postProcess see exec.sh for all the post-processing tools available.
plotData.pywill plot the data generated from the PDE integratorplotCorrelationPINNvsData.pywill show 45 degree plots to check the accuracy of the PINN againsts the PDE integratorplotPINNResult.pywill plot the fields predicted by the best PINNplotPINNResult_movie.pywill plot the field predicted by the best PINN and the correlation plots as movies to check the evolution of the predictions over epochs.plotResidualVariation.pywill display the different losses to ensure that they are properly balanced.
The Bayesian calibration module only needs to call the trained model which can be passed via the command line. See BayesianCalibration_spm/exec_test.sh
The observational data can be generated via makeData.py (see BayesianCalibration_spm/exec_test.sh for usage)
The calibration can be done via cal_nosigma.py (see BayesianCalibration_spm/exec_test.sh for usage)
The likelihood uncertainty sigma is set via bisectional hyperparameter search.
Code formatting and import sorting are done automatically with black and isort.
Fix imports and format : pip install black isort; bash fixFormat.sh
Spelling is checked but not automatically fixed using codespell
This work was authored by the National Renewable Energy Laboratory (NREL), operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. This work was supported by funding from DOE's Vehicle Technologies Office (VTO) and Advanced Scientific Computing Research (ASCR) program. The research was performed using computational resources sponsored by the Department of Energy's Office of Energy Efficiency and Renewable Energy and located at the National Renewable Energy Laboratory. The views expressed in the repository do not necessarily represent the views of the DOE or the U.S. Government.
Recommended citations
@article{hassanaly2024pinn1,
title={PINN surrogate of Li-ion battery models for parameter inference, Part I: Implementation and multi-fidelity hierarchies for the single-particle model},
author={Hassanaly, Malik and Weddle, Peter J and King, Ryan N and De, Subhayan and Doostan, Alireza and Randall, Corey R and Dufek, Eric J and Colclasure, Andrew M and Smith, Kandler},
journal={Journal of Energy Storage},
volume={98},
pages={113103},
year={2024},
publisher={Elsevier}
}
@article{hassanaly2024pinn2,
title={PINN surrogate of Li-ion battery models for parameter inference, Part II: Regularization and application of the pseudo-2D model},
author={Hassanaly, Malik and Weddle, Peter J and King, Ryan N and De, Subhayan and Doostan, Alireza and Randall, Corey R and Dufek, Eric J and Colclasure, Andrew M and Smith, Kandler},
journal={Journal of Energy Storage},
volume={98},
pages={113104},
year={2024},
publisher={Elsevier}
}
@misc{osti_2204976,
title = {PINNSTRIPES (Physics-Informed Neural Network SurrogaTe for Rapidly Identifying Parameters in Energy Systems) [SWR-22-12]},
author = {Hassanaly, Malik and Smith, Kandler and King, Ryan and Weddle, Peter and USDOE Office of Energy Efficiency and Renewable Energy and USDOE Office of Science},
abstractNote = {Energy systems models typically take the form of complex partial differential equations which make multiple forward calculations prohibitively expensive. Fast and data-efficient construction of surrogate models is of utmost importance for applications that require parameter exploration such as design optimization and Bayesian calibration. In presence of a large number of parameters, surrogate models that capture correct dependencies may be difficult to construct with traditional techniques. The issue is addressed here with the formulation of the surrogate model constructed via Physics-Informed Neural Networks (PINN) which capture the dependence with respect to the parameters to estimate, while using a limited amount of data. Since forward evaluations of the surrogate model are cheap, parameter exploration is made inexpensive, even when considering a large number of parameters.},
url = {https://www.osti.gov//servlets/purl/2204976},
doi = {10.11578/dc.20231106.1},
url = {https://www.osti.gov/biblio/2204976}, year = {2023},
month = {10},
note =
}