More flexible `LinearStandardFormCompiler` by bknueven · Pull Request #3949 · Pyomo/pyomo

bknueven · 2026-05-12T16:24:21Z

Fixes # N/A

Summary/Motivation:

LinearStandardFormCompiler is a great tool for rapidly expanding a Pyomo model into the form need for (mixed-integer) linear programming solvers. Many of these solvers also handle ranged (linear) constraints, quadratic constraints, SOS constraints, quadratic objectives, and even sometimes general nonlinear constraints and objectives. This PR would augment the LinearStandardFormCompiler two significant ways:

Functional forms it cannot compile directly are either passed on or ignored and do not raise an error (new option allow_nonlinear and extra_valid_ctypes).
Ranged linear constraints are directly returned as part of the LinearStandardFormInfo when the new option keep_range_constraints=True.

See the usage proof of concept on #3879, which includes these changes.

Changes proposed in this PR:

Add keep_range_constraints to LinearStandardFormCompiler and compile range constraints appropriately (84afd9a, e330102, efc1c74)
Add option for LinearStandardFormCompiler to return non-linear constraints / objectives it encounters instead of raising an error (option allow_nonlinear aa9ce77)
Add option for additional valid ctypes, to be handled by the caller. Useful for SOS constraints (36953d1).
Bugfix for constraints with +/-inf (a13c2e0, c0741a8)
Bugfix: allow TemplateVarRecorder to correctly handle pyomo.kernel variables (6a2c69a)

This PR was created with the help of GitHub Copilot CLI.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

I agree my contributions are submitted under the BSD license.
I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…aints

codecov · 2026-05-14T06:28:09Z

Codecov Report

❌ Patch coverage is 95.87629% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.13%. Comparing base (dccdbdd) to head (436284b).

Files with missing lines	Patch %	Lines
pyomo/repn/plugins/standard_form.py	95.65%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3949      +/-   ##
==========================================
+ Coverage   90.11%   90.13%   +0.01%     
==========================================
  Files         905      905              
  Lines      107502   107579      +77     
==========================================
+ Hits        96878    96964      +86     
+ Misses      10624    10615       -9

Flag	Coverage Δ
builders	`29.11% <4.12%> (-0.02%)`	⬇️
default	`86.47% <95.87%> (?)`
expensive	`35.50% <4.12%> (?)`
linux	`87.62% <95.87%> (-2.00%)`	⬇️
linux_other	`87.62% <95.87%> (+0.01%)`	⬆️
oldsolvers	`28.05% <4.12%> (-0.02%)`	⬇️
osx	`83.00% <95.87%> (+0.01%)`	⬆️
win	`85.55% <95.87%> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jsiirola

I like this approach. I am still digging through the review, but in the interests of getting things moving one big initial question: have you quantified how this change impacts the standard_form performance?

jsiirola · 2026-05-13T17:25:21Z

+        ),
+    )
+    CONFIG.declare(
+        'extra_valid_ctypes',


Potentially rename this ignore_ctypes? Should this be promoted to an ADVANCED_OPTION?

michaelbynum

This is a clever way to enable using the standard form compiler with solvers that support nonlinear expressions. Nice work.

Question - are you actually using kernel, or were you just trying to get tests to pass?

michaelbynum · 2026-05-19T12:26:05Z

+                        offset, linear_index, linear_data, lb, ub = (
+                            template_visitor.expand_expression(obj, obj.template_expr())
+                        )
+                    except InvalidExpressionError:


Is it possible to get an InvalidExpressionError for some reason other than a nonlinear expression?

michaelbynum · 2026-05-19T12:30:30Z

+                        offset, linear_index, linear_data, lb, ub = (
+                            template_visitor.expand_expression(con, con.template_expr())
+                        )
+                    except InvalidExpressionError:


Same question here: Is there a way to get an InvalidExpressionError for a reason other than a nonlinear expression?

michaelbynum · 2026-05-19T12:32:08Z

                linear_index = map(var_recorder.var_order.__getitem__, repn.linear)
                linear_data = repn.linear.values()

+            # Normalize ±inf to None: both kernel constraints and AML


I don't normally see symbols like the +/- symbol here (which show up in several places). Will this cause problems? In other words, do we have any reason to limit ourselves to ASCII (I believe these characters are UTF-8)?

michaelbynum · 2026-05-19T12:52:22Z

+        constraint is not misclassified as a range constraint, and a fully
+        unbounded constraint is skipped rather than emitted as a range row.
+        """
+        import pyomo.kernel as pmo


Do we want new tests for kernel?

michaelbynum · 2026-05-19T12:55:18Z

+        mk = pmo.block()
+        mk.x = pmo.variable()
+        # lb=-inf (unbounded below) → should become a pure ≤ row, not a range row
+        mk.c_ub = pmo.constraint(ub=2.0, body=mk.x)
+        # ub=+inf (unbounded above) → should become a pure ≥ row, not a range row
+        mk.c_lb = pmo.constraint(lb=-3.0, body=mk.x)
+        # Explicit finite range → should still be a range row
+        mk.c_rng = pmo.constraint((-1.0, mk.x, 4.0))


I was surprised that I didn't see any inf bounds here given the name of the test and the docstring for the function. This is not necessarily a bad test, but I don't see how it tests inf bounds.

michaelbynum · 2026-05-19T12:57:56Z

+        when ``allow_nonlinear=False`` (the default).  When
+        ``allow_nonlinear=True``, holds the list of objectives with nonlinear
+        terms that were omitted from the compiled matrices (may be empty).
+


Should this class also contain a list of the active ctypes that it found but ignored (when extra_valid_ctypes is used)?

michaelbynum · 2026-05-19T13:01:28Z

        except AttributeError:
-            # Note that this only works for the AML, as kernel does not
-            # provide a parent_component()
-            _iter = (None, var)


I don't understand how this worked previously. Wouldn't an error get raised below because there was no tuple to unpack?

michaelbynum · 2026-05-19T13:03:18Z

I don't see anything that could impact performance unless allow_nonlinear is set to True. Well, there is the try/except statement in repn/util - I don't know if that could impact things or not.

Bernard Knueven added 12 commits May 11, 2026 12:27

add keep_range_constraints to LinearStandardFormCompiler

84afd9a

NFC: formatting

28e3529

allow LinearStandardFormCompiler to return unhandled nonlinear constr…

aa9ce77

…aints

cleanup range constraints

e330102

update standard form test for better range constraint handling

efc1c74

NFC: black on standard_form.py

ea34736

allow SOS constraints (and other ctypes) to pass for standard form

36953d1

NFC: black

e431ec9

handle kernel range constraints in LSFC

a13c2e0

bugfix: standard form normalizes constraints with +/-inf bounds

c0741a8

fix bug pyomo.kernel variabls in TemplateVarRecorder

6a2c69a

update LSFI to be self-consistent

ca7f765

bknueven mentioned this pull request May 12, 2026

Performance enhancements for XpressDirect / XpressPersistent for large LPs / MIPs #3879

Open

Merge branch 'main' into flexible-linear-standard-repn

436284b

jsiirola added the AT: PRE-TEST INSPECTED label May 14, 2026

pyomo-autotest removed the AT: PRE-TEST INSPECTED label May 14, 2026

jsiirola reviewed May 18, 2026

View reviewed changes

michaelbynum reviewed May 19, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More flexible `LinearStandardFormCompiler`#3949

More flexible `LinearStandardFormCompiler`#3949
bknueven wants to merge 13 commits into
Pyomo:mainfrom
bknueven:flexible-linear-standard-repn

bknueven commented May 12, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 14, 2026 •

edited

Loading

Uh oh!

jsiirola left a comment

Uh oh!

jsiirola May 13, 2026

Uh oh!

michaelbynum left a comment

Uh oh!

michaelbynum May 19, 2026

Uh oh!

michaelbynum May 19, 2026

Uh oh!

michaelbynum May 19, 2026

Uh oh!

michaelbynum May 19, 2026

Uh oh!

michaelbynum May 19, 2026

Uh oh!

michaelbynum May 19, 2026

Uh oh!

michaelbynum May 19, 2026

Uh oh!

michaelbynum commented May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

bknueven commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Fixes # N/A

Summary/Motivation:

Changes proposed in this PR:

Legal Acknowledgement

Uh oh!

codecov Bot commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

jsiirola left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelbynum left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelbynum commented May 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

bknueven commented May 12, 2026 •

edited

Loading

codecov Bot commented May 14, 2026 •

edited

Loading