Implementing multistart version of theta_est using multiple sampling methods

[sscini]

Fixes # .

Summary/Motivation:

Currently, the optimization is only done from a single initial value. This implementation adds the ability to specify multiple initial values using selected sampling techniques: from a random uniform distribution, using Latin Hypercube Sampling, or using Sobol Quasi-Monte Carlo sampling.

Changes proposed in this PR:

• All changes made adding pseudocode in comments
• Added inputs needed for multistart simulation
• Added a function to generate points using the selected method
• Added theta_est_multistart to work for the multistart process

TODO before converting from draft:

• Receive feedback from collaborators on logical setup
• Convert finalized pseudocode
• Test and debug
• Confirm function with examples

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[sscini]

@djlaky @adowling2 Please provide early feedback.

[sscini]

Dynamic saving using flush, add.

[adowling2]

Notes from our in-person discussion/informal code review

[sscini]

Extend existing tests for parmest to include multistart, add.

[sscini]

Models provided need to include bounds, add exception

[adowling2]

Here are some more comments for you to consider are you continue to refine this.

[adowling2]

You probably already know this, but you will need to check all the errors are raised when expected.

[sscini]

Now that _Q_opt has been finalized and merged, going to revisit this, make adjustments as needed from final _Q_opt and regularization revisions, and then will make a regular PR.

[sscini]

@mrmundt Hi Miranda! I had a quick testing question if you can help clarify something. It looks like if a test has been marked expensive, it does not count as covered (according to coverage reports from individual checks). If this is as intended, is the goal to change those tests long-term? Curious your thoughts. Thanks!

[mrmundt]

@mrmundt Hi Miranda! I had a quick testing question if you can help clarify something. It looks like if a test has been marked expensive, it does not count as covered (according to coverage reports from individual checks). If this is as intended, is the goal to change those tests long-term? Curious your thoughts. Thanks!

HELLO! Okay so, yes, expensive tests aren't run in the standard suite. We have a specific nightly job that exercises them on Jenkins (which does then upload a coverage report), but it only runs on main. The purpose behind that is, because they are expensive and we already have such a large test suite, we don't turn those on for individual PR testing in order to speed up the suite.

[jsiirola]

(@mrmundt: actually, the Jenkins PR tests do run the expensive tests ... we need to have the main and PR tests run the same things in order to get meaningful coverage information)

[mrmundt]

(@mrmundt: actually, the Jenkins PR tests do run the expensive tests ... we need to have the main and PR tests run the same things in order to get meaningful coverage information)

Oh is that new? Because that wasn't the case before. ALRIGHT. Then it seems like it's a bug that the coverage reports aren't showing coverage on the expensive suite, too.

EDIT: AHA! Right, they only run on Jenkins, and yours doesn't have the Jenkins job yet :P

[codecov]

Codecov Report

❌ Patch coverage is 70.71823% with 53 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.11%. Comparing base (e099f64) to head (4589624).

Files with missing lines	Patch %	Lines
pyomo/contrib/parmest/parmest.py	82.83%	23 Missing ⚠️
...mest/examples/reactor_design/multistart_example.py	31.81%	15 Missing ⚠️
...mest/examples/rooney_biegler/multistart_example.py	28.57%	15 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3575      +/-   ##
==========================================
- Coverage   90.14%   90.11%   -0.04%     
==========================================
  Files         917      919       +2     
  Lines      109261   109430     +169     
==========================================
+ Hits        98494    98609     +115     
- Misses      10767    10821      +54     

Flag	Coverage Δ
builders	29.08% <2.76%> (-0.04%)	⬇️
default	86.10% <60.22%> (?)
expensive	35.57% <6.07%> (?)
linux	87.65% <70.71%> (-2.01%)	⬇️
linux_other	87.65% <70.71%> (-0.03%)	⬇️
oldsolvers	28.01% <1.65%> (-0.05%)	⬇️
osx	83.08% <60.22%> (-0.04%)	⬇️
win	85.39% <60.22%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sscini]

@adowling2 @slilonfe5 @smondal13 I think this is ready to go. Another pair of eyes on the finalized would be great. Thanks!