Skip to content

Drop a within-entity constant when an intercept is present - #730

Open
shaneraphel wants to merge 1 commit into
bashtage:mainfrom
shaneraphel:drop-absorbed-constant
Open

shaneraphel wants to merge 1 commit into
bashtage:mainfrom
shaneraphel:drop-absorbed-constant

Conversation

@shaneraphel

@shaneraphel shaneraphel commented Sep 27, 2026 •

Copy link
Copy Markdown

Summary

With entity effects and an intercept, a regressor that does not vary within an entity is demeaned to zero and then restored as its grand mean. not_absorbed residualizes that column on the intercept. Exact arithmetic leaves nothing, but the eigenvalue tolerance is taken relative to the remaining roundoff, so the column is kept. On the bundled wage panel, education is constant for every worker. lwage ~ 1 + educ + EntityEffects with drop_absorbed=True reported 0.13914557037073996. Replacing education by education minus 12 reported -0.36450348040229325. Both p-values were 0. The same model with union included already dropped education and kept union at 0.07468459435240984. drop_absorbed=False already raised AbsorbingEffectError.

The extra check compares the residual norm with the column's own norm, and it returns only the intercept when every non-constant column vanishes. A regression that still has a within-entity regressor uses the existing rank test. After the change both encodings drop the time-invariant column and return the same intercept, 1.6491471921100918.

Fixes #728.

This patch was prepared with the help of an AI assistant. I compared the two encodings of education on the bundled wage panel, and the regression that also includes union.

Test plan

  • pytest linearmodels/tests/panel/test_panel_ols.py::test_drop_absorbed_constant_when_regressor_is_time_invariant linearmodels/tests/panel/test_panel_ols.py::test_fully_absorbed linearmodels/tests/panel/test_panel_ols.py::test_absorbed (4 passed on this branch)
  • Wage panel: both encodings keep only the intercept 1.6491471921100918
  • union stays at 0.07468459435240984 and education is still dropped
  • drop_absorbed=False still raises AbsorbingEffectError

Entity effects remove it, and adding the grand mean back leaves only roundoff. That roundoff was enough to keep the column, so two encodings of the same variable produced different coefficients.
@codecov

codecov Bot commented Sep 27, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.53%. Comparing base (ec0f288) to head (c383283).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #730      +/-   ##
==========================================
- Coverage   99.54%   99.53%   -0.02%     
==========================================
  Files         101      101              
  Lines       17450    17476      +26     
  Branches     1433     1435       +2     
==========================================
+ Hits        17371    17394      +23     
- Misses         29       31       +2     
- Partials       50       51       +1     
Flag Coverage Δ
adder 99.51% <100.00%> (-0.02%) ⬇️
subtractor 99.51% <100.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PanelOLS can retain fully absorbed regressors when a constant is present

1 participant