Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding basic parameter sweep tool #1284

Merged
merged 33 commits into from
Feb 22, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
d994c81
Initial work on parameter sweep
andrewlee94 Nov 1, 2023
5166e73
Merge branch 'main' of https://github.com/IDAES/idaes-pse into parame…
andrewlee94 Nov 7, 2023
1f5e33a
Fixing config tests
andrewlee94 Nov 7, 2023
d3e711c
Working on testing parameter sweep base class
andrewlee94 Nov 7, 2023
9e80e17
Finishing parameter sweep base file
andrewlee94 Nov 7, 2023
98a2503
Adding basic sequential runner
andrewlee94 Nov 7, 2023
bffa953
Deprecate old convergence tester tools
andrewlee94 Nov 7, 2023
fc92607
Merging main
andrewlee94 Nov 10, 2023
bb8c2a6
Working on ConvergenceAnalysis
andrewlee94 Nov 10, 2023
df3b2f3
Adding methods to compare to baseline
andrewlee94 Nov 12, 2023
f7fe78f
Adding doc strings
andrewlee94 Nov 12, 2023
89c1f50
Fixing spelling
andrewlee94 Nov 12, 2023
0a881ad
Adding parameterization to parameter sweep callbacks
andrewlee94 Nov 13, 2023
bf367cf
Fixing conflicts
andrewlee94 Nov 17, 2023
f6ef8ce
Merging main
andrewlee94 Nov 22, 2023
14ba342
Renaming recourse and switching to take solver object instead of name
andrewlee94 Nov 22, 2023
a1ccd98
Addressing comments on parameter sweep
andrewlee94 Nov 22, 2023
7b99a10
Updating convergence tester
andrewlee94 Nov 22, 2023
269a097
Adding progress bar
andrewlee94 Nov 22, 2023
79bb2fa
Fixing pyling issues
andrewlee94 Nov 22, 2023
4e3179c
Merging main
andrewlee94 Dec 8, 2023
2264ce2
Changing how solver is handled
andrewlee94 Dec 8, 2023
f0acaf5
Rename collect_results to build_results for consistency with WaterTAP
andrewlee94 Dec 8, 2023
900aecb
Merge branch 'main' of https://github.com/IDAES/idaes-pse into parame…
andrewlee94 Dec 15, 2023
4d09619
Merging main
andrewlee94 Feb 6, 2024
b738d6f
Merge branch 'main' into parameter_sweep
ksbeattie Feb 9, 2024
01403b6
Merge branch 'main' of https://github.com/IDAES/idaes-pse into parame…
andrewlee94 Feb 20, 2024
1d4997d
Addressing most PR comments
andrewlee94 Feb 20, 2024
42912bc
Fixing doc string formatting
andrewlee94 Feb 21, 2024
cf9bf92
Update idaes/core/util/parameter_sweep.py
andrewlee94 Feb 21, 2024
3f07eec
Fixing typo in test regex
andrewlee94 Feb 21, 2024
dfcfad0
Merge branch 'main' of https://github.com/IDAES/idaes-pse into parame…
andrewlee94 Feb 21, 2024
f697d85
Renaming ConvergenceAnalysis to IpoptConvergenceAnalysis
andrewlee94 Feb 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
230 changes: 141 additions & 89 deletions idaes/core/util/convergence/convergence_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
- a Pyomo solver with appropriate options

The module executes convergence evaluation in two steps. In the first step, a
json file is created that containsa set of points sampled from the provided
json file is created that contains a set of points sampled from the provided
inputs. This step only needs to be done once - up front. The second step, which
should be executed any time there is a major code change that could impact the
model, takes that set of sampled points and solves the model at each of the
Expand Down Expand Up @@ -79,6 +79,7 @@ class from ConvergenceEvaluation, and implement three methods:
from pyomo.core import Param, Var
from pyomo.common.log import LoggingIntercept
from pyomo.environ import check_optimal_termination
from pyomo.common.deprecation import deprecated

# idaes
import idaes.core.util.convergence.mpi_utils as mpiu
Expand All @@ -92,6 +93,11 @@ class from ConvergenceEvaluation, and implement three methods:
convergence_classes = {}


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def register_convergence_class(name):
def _register_convergence_class(cls):
if name in convergence_classes:
Expand All @@ -102,6 +108,11 @@ def _register_convergence_class(cls):
return _register_convergence_class


@deprecated(
msg="This class has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
class ConvergenceEvaluationSpecification(object):
"""
Object for defining sample points for convergence evaluations.
Expand All @@ -121,28 +132,27 @@ def add_sampled_input(
(with given mean and standard deviation)
truncated to the values given by lower and upper bounds

Parameters
----------
name : str
The name of the input.
pyomo_path : str
A string representation of the path to the variable or parameter to
be sampled. This string will be executed to retrieve the Pyomo
component.
lower : float
A lower bound on the input variable or parameter.
upper : float
An upper bound on the input variable or parameter
mean : float
The mean value to use when generating normal distribution samples
std : float
The standard deviation to use when generating normal distribution samples
distribution : str
The Distribution type {"normal", "uniform"}
Args:
name - str
The name of the input.
pyomo_path - str
A string representation of the path to the variable or parameter to
be sampled. This string will be executed to retrieve the Pyomo
component.
lower - float
A lower bound on the input variable or parameter.
upper - float
An upper bound on the input variable or parameter
mean - float
The mean value to use when generating normal distribution samples
std - float
The standard deviation to use when generating normal distribution samples
distribution - str
The Distribution type {"normal", "uniform"}

Returns:
None

Returns
-------
N/A
"""
# ToDo: put some error checking here ... Maybe we should have the model
# ToDo: already? Can use to check if the pyomo_path is valid? check if
Expand All @@ -161,6 +171,11 @@ def inputs(self):
return self._inputs


@deprecated(
msg="This class has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
class ConvergenceEvaluation:
"""
Object for running convergence evaluations.
Expand Down Expand Up @@ -196,11 +211,10 @@ def get_initialized_model(self):
values of parameters or variables according to the sampling
specifications.

Returns
-------
Pyomo model : return a Pyomo model object that is initialized and
ready to solve. This is the model object that will be
used in the evaluation.
Returns:
Pyomo model - return a Pyomo model object that is initialized and
ready to solve. This is the model object that will be
used in the evaluation.
"""
raise NotImplementedError(
"Not implemented in the base class. This"
Expand All @@ -214,9 +228,8 @@ def get_solver(self):

Users may overload this to use a custom solver or options if required.

Returns
-------
Pyomo solver
Returns:
Pyomo solver

"""
return get_solver()
Expand Down Expand Up @@ -397,26 +410,22 @@ def _run_ipopt_with_stats(model, solver, max_iter=500, max_cpu_time=120):
"""
Run the solver (must be ipopt) and return the convergence statistics

Parameters
----------
model : Pyomo model
The pyomo model to be solved

solver : Pyomo solver
The pyomo solver to use - it must be ipopt, but with whichever options
are preferred

max_iter : int
The maximum number of iterations to allow for ipopt
Args:
model - Pyomo model
The pyomo model to be solved
solver - Pyomo solver
The pyomo solver to use - it must be ipopt, but with whichever options
are preferred
max_iter - int
The maximum number of iterations to allow for ipopt
max_cpu_time - int
The maximum cpu time to allow for ipopt (in seconds)

max_cpu_time : int
The maximum cpu time to allow for ipopt (in seconds)
Returns:
Returns a tuple with (solve status object, bool (solve successful or
not), number of iters, number of iters in restoration, number of iters with regularization,
solve time)

Returns
-------
Returns a tuple with (solve status object, bool (solve successful or
not), number of iters, number of iters in restoration, number of iters with regularization,
solve time)
"""
# ToDo: Check that the "solver" is, in fact, IPOPT

Expand Down Expand Up @@ -510,25 +519,30 @@ def _set_model_parameters_from_sample(model, inputs, sample_point):
)


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def generate_samples(eval_spec, n_points, seed=None):
"""
Samples the space of the inputs defined in the eval_spec, and creates an
OrderedDict with all the points to be used in executing a convergence
evaluation

Parameters
----------
eval_spec : ConvergenceEvaluationSpecification
The convergence evaluation specification object that we would like to
sample
n_points : int
The total number of points that should be created
seed : int or None
The seed to be used when generating samples. If set to None, then the
seed is not set
Returns
-------
Args:
eval_spec - ConvergenceEvaluationSpecification
The convergence evaluation specification object that we would like to
sample
n_points - int
The total number of points that should be created
seed - int or None
The seed to be used when generating samples. If set to None, then the
seed is not set

Returns:
OrderedDict of samples

"""
if seed is not None:
np.random.seed(seed)
Expand All @@ -549,6 +563,11 @@ def generate_samples(eval_spec, n_points, seed=None):
return samples


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def write_sample_file(
eval_spec, filename, convergence_evaluation_class_str, n_points, seed=None
):
Expand All @@ -557,25 +576,25 @@ def write_sample_file(
json file with all the points to be used in executing a convergence
evaluation

Parameters
----------
filename : str
The filename for the json file that will be created containing all the
points to be run
eval_spec : ConvergenceEvaluationSpecification
The convergence evaluation specification object that we would like to
sample
convergence_evaluation_class_str : str
Python string that identifies the convergence evaluation class for this
specific evaluation. This is usually in the form of module.class_name.
n_points : int
The total number of points that should be created
seed : int or None
The seed to be used when generating samples. If set to None, then the
seed is not set
Returns
-------
N/A
Args:
filename - str
The filename for the json file that will be created containing all the
points to be run
eval_spec - ConvergenceEvaluationSpecification
The convergence evaluation specification object that we would like to
sample
convergence_evaluation_class_str - str
Python string that identifies the convergence evaluation class for this
specific evaluation. This is usually in the form of module.class_name.
n_points - int
The total number of points that should be created
seed - int or None
The seed to be used when generating samples. If set to None, then the
seed is not set

Returns:
None

"""
# build the samples
samples = generate_samples(eval_spec, n_points, seed)
Expand All @@ -592,6 +611,11 @@ def write_sample_file(
json.dump(jsondict, fd, indent=3)


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def run_convergence_evaluation_from_sample_file(sample_file):
"""
Run convergence evaluation using specified sample file.
Expand Down Expand Up @@ -624,6 +648,11 @@ def run_convergence_evaluation_from_sample_file(sample_file):
return run_convergence_evaluation(jsondict, conv_eval)


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def run_single_sample_from_sample_file(sample_file, name):
"""
Run single convergence evaluation from sample in provided file.
Expand Down Expand Up @@ -657,6 +686,11 @@ def run_single_sample_from_sample_file(sample_file, name):
return run_single_sample(jsondict, conv_eval, name)


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def run_single_sample(sample_file_dict, conv_eval, name):
"""
Run single sample from dict and return IPOPT stats.
Expand All @@ -676,23 +710,26 @@ def run_single_sample(sample_file_dict, conv_eval, name):
return _run_ipopt_with_stats(model, solver)


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def run_convergence_evaluation(sample_file_dict, conv_eval):
"""
Run convergence evaluation and generate the statistics based on information
in the sample_file.

Parameters
----------
sample_file_dict : dict
Dictionary created by ConvergenceEvaluationSpecification that contains
the input and sample point information
Args:
sample_file_dict - dict
Dictionary created by ConvergenceEvaluationSpecification that contains
the input and sample point information
conv_eval - ConvergenceEvaluation
The ConvergenceEvaluation object that should be used

conv_eval : ConvergenceEvaluation
The ConvergenceEvaluation object that should be used
Returns:
None

Returns
-------
N/A
"""
inputs = sample_file_dict["inputs"]
samples = sample_file_dict["samples"]
Expand Down Expand Up @@ -754,6 +791,11 @@ def run_convergence_evaluation(sample_file_dict, conv_eval):
return inputs, samples, global_results


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def generate_baseline_statistics(
conv_eval, n_points: int, seed: int = None, display: bool = True
):
Expand Down Expand Up @@ -818,6 +860,11 @@ def generate_baseline_statistics(
return jsondict


@deprecated(
msg="This function has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
def save_convergence_statistics(
inputs, results, dmf=None, display=True, json_path=None, report_path=None
):
Expand All @@ -838,6 +885,11 @@ def save_convergence_statistics(
return s


@deprecated(
msg="This class has been deprecated in favor of the new Parameter Sweep "
"tools and may be removed in a future release.",
version="2.3.0",
)
class Stats(object):
def __init__(self, inputs=None, results=None, from_dict=None, from_json=None):
"""A convergence stats and results object. This class stores the
Expand Down
Loading
Loading