Editing Multicollinearity (section)

=== Solutions ===
Numerical problems in estimating can be solved by applying standard techniques from [[linear algebra]] to estimate the equations more precisely:

# [[Standard score|'''Standardizing''']] '''predictor variables.''' Working with polynomial terms (e.g. <math>x_1</math>, <math>x_1^2</math>), including interaction terms (i.e., <math>x_1 \times x_2</math>) can cause multicollinearity. This is especially true when the variable in question has a limited range. Standardizing predictor variables will eliminate this special kind of multicollinearity for polynomials of up to 3rd order.<ref>{{Cite web |title=12.6 - Reducing Structural Multicollinearity {{!}} STAT 501 |url=https://newonlinecourses.science.psu.edu/stat501/lesson/12/12.6 |access-date=2019-03-16 |website=newonlinecourses.science.psu.edu}}</ref>
#* For higher-order polynomials, an [[Orthogonal polynomials|orthogonal polynomial]] representation will generally fix any collinearity problems.<ref name=":4">{{Cite web |title=Computational Tricks with Turing (Non-Centered Parametrization and QR Decomposition) |url=https://storopoli.io/Bayesian-Julia/pages/12_Turing_tricks/#qr_decomposition |access-date=2023-09-03 |website=storopoli.io}}</ref> However, polynomial regressions are [[Runge's phenomenon|generally unstable]], making them unsuitable for [[nonparametric regression]] and inferior to newer methods based on [[smoothing spline]]s, [[LOESS]], or [[Gaussian process]] regression.<ref>{{Cite journal |last1=Gelman |first1=Andrew |last2=Imbens |first2=Guido |date=2019-07-03 |title=Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs |url=https://www.tandfonline.com/doi/full/10.1080/07350015.2017.1366909 |journal=Journal of Business & Economic Statistics |language=en |volume=37 |issue=3 |pages=447–456 |doi=10.1080/07350015.2017.1366909 |issn=0735-0015|url-access=subscription }}</ref>
# '''Use an [[QR decomposition|orthogonal representation]] of the data'''.<ref name=":4" /> Poorly-written statistical software will sometimes fail to converge to a correct representation when variables are strongly correlated. However, it is still possible to rewrite the regression to use only uncorrelated variables by performing a [[change of basis]].
#* For polynomial terms in particular, it is possible to rewrite the regression as a function of uncorrelated variables using [[orthogonal polynomials]].