Editing Linear trend estimation

{{Short description|Statistical technique to aid interpretation of data}}
{{broader|Curve fitting}}
{{Multiple issues|
{{More footnotes|date=July 2019}}
{{Tone|date=September 2023}}
{{Technical|date=December 2023}}
}}

'''Linear trend estimation''' is a [[statistics|statistical]] technique used to analyze [[data]] patterns. [[Data]] patterns, or trends, occur when the [[information]] gathered tends to increase or decrease over time or is influenced by changes in an external factor. Linear trend estimation essentially creates a straight line on a [[Graph theory|graph]] of [[data]] that models the general direction that the [[data]] is heading.  

==Fitting a trend: Least-squares==
Given a set of [[data]], there are a variety of [[Function (mathematics)|functions]] that can be chosen to fit the data. The simplest function is a [[Line (geometry)|straight line]] with the dependent variable (typically the measured data) on the vertical axis and the independent variable (often time) on the horizontal axis.

The [[least-squares]] fit is a common method to fit a straight line through the data. This method [[Optimization problem|minimizes]] the sum of the squared errors in the data series <math>y</math>. Given a set of points in time <math>t</math> and data values <math>y_t</math> observed for those points in time, values of <math>\hat a</math> and <math>\hat b</math> are chosen to minimize the sum of squared errors

:<math>\sum_t \left[ y_t - \left( \hat{a}t + \hat{b} \right) \right]^2</math>.

This formula first calculates the difference between the observed data <math>y_t</math> and the estimate <math>(\hat{a}t + \hat{b})</math>, the difference at each data point is squared, and then added together, giving the "sum of squares" measurement of error. The values of <math>\hat{a}</math> and <math>\hat{b}</math> derived from the data parameterize the simple linear estimator <math>\hat{y} = \hat{a} x + \hat{b}</math>. The term "trend" refers to the slope <math>\hat{a}</math> in the least squares estimator.

==Data as trend and noise==
To analyze a (time) series of data, it can be assumed that it may be represented as trend plus noise:

:<math>y_t = at + b + e_t\,</math>

where <math>a</math> and <math>b</math> are unknown constants and the <math>e</math>'s are randomly distributed [[errors and residuals|errors]]. If one can reject the null hypothesis that the errors are [[unit root|non-stationary]], then the non-stationary series <math>\{y_t\}</math> is called [[trend-stationary process|trend-stationary]]. The least-squares method assumes the errors are independently distributed with a normal distribution. If this is not the case, hypothesis tests about the unknown parameters <math>a</math> and <math>b</math> may be inaccurate. It is simplest if the <math>e</math>'s all have the same distribution, but if not (if some have [[heteroskedasticity|higher variance]], meaning that those data points are effectively less certain), then this can be taken into account during the least-squares fitting by weighting each point by the inverse of the variance of that point.

Commonly, where only a single time series exists to be analyzed, the variance of the <math>e</math>'s is estimated by fitting a trend to obtain the estimated parameter values <math>\hat a</math> and <math>\hat b,</math> thus allowing the predicted values 
:<math>\hat y =\hat at+\hat b</math>

to be subtracted from the data <math>y_t</math> (thus ''detrending'' the data), leaving the [[errors and residuals|residuals]] <math>\hat e_t</math> as the ''detrended data'', and estimating the variance of the <math>e_t</math>'s from the residuals — this is often the only way of estimating the variance of the <math>e_t</math>'s.

Once the "noise" of the series is known, the significance of the trend can be assessed by making the [[null hypothesis]] that the trend, <math>a</math>, is not different from 0. From the above discussion of trends in random data with known [[variance]], the distribution of calculated trends is to be expected from random (trendless) data. If the estimated trend, <math>\hat a</math>, is larger than the critical value for a certain [[significance level]], then the estimated trend is deemed significantly different from zero at that significance level, and the null hypothesis of a zero underlying trend is rejected.

The use of a linear trend line has been the subject of criticism, leading to a search for alternative approaches to avoid its use in model estimation. One of the alternative approaches involves [[unit root]] tests and the [[cointegration]] technique in econometric studies.

The estimated coefficient associated with a linear trend variable such as time is interpreted as a measure of the impact of a number of unknown or known but immeasurable factors on the dependent variable over one unit of time. Strictly speaking, this interpretation is applicable for the estimation time frame only. Outside of this time frame, it cannot be determined how these immeasurable factors behave both qualitatively and quantitatively. 

Research results by mathematicians, statisticians, econometricians, and economists have been published in response to those questions. For example, detailed notes on the meaning of linear time trends in the regression model are given in Cameron (2005);<ref name=":0">{{cite news|url=http://highered.mcgraw-hill.com/sites/dl/free/0077104285/160071/Chapter_7.pdf|title=Making Regression More Useful II: Dummies and Trends|access-date=June 17, 2012}}</ref> Granger, Engle, and many other econometricians have written on stationarity, unit root testing, co-integration, and related issues (a summary of some of the works in this area can be found in an information paper<ref>{{cite news|url=http://www.kva.se/Documents/Priser/Nobel/2003/sciback_ek_en_03.pdf|title=The Royal Swedish Academy of Sciences|date=8 October 2003|access-date=June 17, 2012}}</ref> by the Royal Swedish Academy of Sciences (2003)); and Ho-Trieu & Tucker (1990) have written on logarithmic time trends with results indicating linear time trends are special cases of [[cycle (sequence)|cycle]]s.

=== Noisy time series ===
It is harder to see a trend in a noisy time series. For example, if the true series is 0, 1, 2, 3, all plus some independent normally distributed "noise" ''e'' of [[standard deviation]]{{nbsp}}''E'', and a sample series of length 50 is given, then if ''E''{{nbsp}}={{nbsp}}0.1, the trend will be obvious; if ''E''{{nbsp}}={{nbsp}}100, the trend will probably be visible; but if ''E''{{nbsp}}={{nbsp}}10000, the trend will be buried in the noise.

Consider a concrete example, such as the [[global surface temperature]] record of the past 140 years as presented by the [[Intergovernmental Panel on Climate Change|IPCC]].<ref name=":1">{{cite news|url=http://www.grida.no/publications/other/ipcc_tar/?src=/climate/ipcc_tar/wg1/figspm-1.htm|title=IPCC Third Assessment Report – Climate Change 2001 – Complete online versions|access-date=June 17, 2012|url-status=dead|archive-url=https://web.archive.org/web/20091120181301/http://www.grida.no/publications/other/ipcc_tar/?src=%2Fclimate%2Fipcc_tar%2Fwg1%2Ffigspm-1.htm|archive-date=November 20, 2009}}</ref> The interannual variation is about 0.2{{nbsp}}°C, and the trend is about 0.6{{nbsp}}°C over 140 years, with 95% confidence limits of 0.2{{nbsp}}°C (by coincidence, about the same value as the interannual variation). Hence, the trend is statistically different from 0. However, as noted elsewhere,<ref name=":2" /> this time series doesn't conform to the assumptions necessary for least-squares to be valid.

==Goodness of fit (''r''-squared) and trend==
[[Image:Random-data-plus-trend-r2.png|thumb|right|Illustration of the effect of filtering on ''r''<sup>2</sup>. Black&nbsp;=&nbsp;unfiltered data; red&nbsp;=&nbsp;data averaged every 10 points; blue&nbsp;=&nbsp;data averaged every 100 points. All have the same trend, but more filtering leads to higher ''r''<sup>2</sup> of fitted trend line.]]

The least-squares fitting process produces a value, [[Coefficient of determination|r-squared]] (''r''<sup>2</sup>), which is 1 minus the ratio of the variance of the [[Errors and residuals|residuals]] to the variance of the dependent variable. It says what fraction of the variance of the data is explained by the fitted trend line. It does '''not''' relate to the [[statistical significance]] of the trend line (see graph); the statistical significance of the trend is determined by its [[t-statistic]]. Often, filtering a series increases ''r''<sup>2</sup> while making little difference to the fitted trend.

==Advanced models==
Thus far, the data have been assumed to consist of the trend plus noise, with the noise at each data point being [[independent and identically distributed random variables]] with a normal distribution. Real data (for example, climate data) may not fulfill these criteria. This is important, as it makes an enormous difference to the ease with which the statistics can be analyzed so as to extract maximum information from the data series. If there are other non-linear effects that have a [[correlation]] to the independent variable (such as cyclic influences), the use of least-squares estimation of the trend is not valid. Also, where the variations are significantly larger than the resulting straight line trend, the choice of start and end points can significantly change the result. That is, the model is mathematically [[Statistical model specification|misspecified]]. Statistical inferences (tests for the presence of a trend, confidence intervals for the trend, etc.) are invalid unless departures from the standard assumptions are properly accounted for, for example, as follows:

*Dependence: autocorrelated time series might be modelled using [[autoregressive moving average model]]s.
*Non-constant variance: in the simplest cases, [[weighted least squares]] might be used.
*Non-normal distribution for errors: in the simplest cases, a [[generalised linear model|generalized linear model]] might be applicable.
*[[Unit root]]: taking first (or occasionally second) differences of the data, with the level of differencing being identified through various unit root tests.<ref name=":2">{{cite book|url=https://www.otexts.org/fpp/8/1|title=Forecasting: principles and practice|date=20 September 2014|access-date=May 17, 2015}}</ref>
In [[R (programming language)|R]], the linear trend in data can be estimated by using the 'tslm' function of the 'forecast' package.

==Trends in clinical data==
Medical and [[biomedical]] studies often seek to determine a link between sets of data, such as of a clinical or scientific metric in three different diseases. But data may also be linked in time (such as change in the effect of a drug from baseline, to month 1, to month 2), or by an external factor that may or may not be determined by the researcher and/or their subject (such as no pain, mild pain, moderate pain, or severe pain). In these cases, one would expect the effect test statistic (e.g., influence of a [[statin]] on levels of [[cholesterol]], an [[analgesic]] on the degree of pain, or increasing doses of different strengths of a drug on a measurable index, i.e. a dose - response effect) to change in direct order as the effect develops. Suppose the mean level of cholesterol before and after the prescription of a statin falls from 5.6 [[mmol/L]] at baseline to 3.4 mmol/L at one month and to 3.7 mmol/L at two months. Given sufficient power, an [[Analysis of variance| ANOVA (analysis of variance)]] would most likely find a significant fall at one and two months, but the fall is not linear. Furthermore, a post-hoc test may be required. An alternative test may be a repeated measures (two way) ANOVA or [[Friedman test]], depending on the nature of the data. Nevertheless, because the groups are ordered, a standard ANOVA is inappropriate. Should the cholesterol fall from 5.4 to 4.1 to 3.7, there is a clear linear trend. The same principle may be applied to the effects of allele/[[genotype frequency]], where it could be argued that a [[single-nucleotide polymorphism]] in nucleotides XX, XY, YY are in fact a trend of no Y's, one Y, and then two Y's.<ref name=":1" />

The mathematics of linear trend estimation is a variant of the standard ANOVA, giving different information, and would be the most appropriate test if the researchers hypothesize a trend effect in their test statistic. One example is levels of serum [[trypsin]] in six groups of subjects ordered by age decade (10–19 years up to 60–69 years). Levels of trypsin (ng/mL) rise in a direct linear trend of 128, 152, 194, 207, 215, 218 (data from Altman). Unsurprisingly, a 'standard' ANOVA gives ''p''&nbsp;<&nbsp;0.0001, whereas linear trend estimation gives ''p''&nbsp;=&nbsp;0.00006. Incidentally, it could be reasonably argued that as age is a natural continuously variable index, it should not be categorized into decades, and an effect of age and serum trypsin is sought by correlation (assuming the raw data is available). A further example is of a substance measured at four time points in different groups:
{| class="wikitable"
|+
!#
!mean
!SD
|-
|1
|1.6
|0.56
|-
|2
|1.94
|0.75
|-
|3
|2.22
|0.66
|-
|4
|2.40
|0.79
|}
This is a clear trend. ANOVA gives ''p''&nbsp;=&nbsp;0.091, because the overall variance exceeds the means, whereas linear trend estimation gives ''p''&nbsp;=&nbsp;0.012. However, should the data have been collected at four time points in the same individuals, linear trend estimation would be inappropriate, and a two-way (repeated measures) ANOVA would have been applied.

==See also==
*[[Estimation statistics|Estimation]]
*[[Extrapolation]]
*[[Forecasting]]
*[[Least squares]]
*[[Least-squares spectral analysis]]
*[[Line fitting]]
*[[Prediction interval]]
*[[Regression analysis]]

==Notes==
{{Reflist}}

==References==
*{{cite journal|last1=Bianchi|first1=M.|last2=Boyle|first2=M.|last3=Hollingsworth|first3=D.|year=1999|title=A comparison of methods for trend estimation|journal=[[Applied Economics Letters]]|volume=6|issue=2|pages=103–109|doi=10.1080/135048599353726}}
*{{cite book|last=Cameron|first=S.|year=2005|title=Econometrics|chapter=Making Regression Analysis More Useful, II|publisher=McGraw Hill Higher Education|location=Maidenhead|isbn=0077104285|pages=171–198}}
*{{cite journal|last=Chatfield|first=C.|year=1993|title=Calculating Interval Forecasts|journal=Journal of Business and Economic Statistics|volume=11|issue=2|pages=121–135|doi=10.1080/07350015.1993.10509938}}
*{{cite journal|last1=Ho-Trieu|first1=N. L.|last2=Tucker|first2=J.|year=1990|title=Another note on the use of a logarithmic time trend|journal=Review of Marketing and Agricultural Economics|volume=58|issue=1|pages=89–90|doi=10.22004/ag.econ.12288}}
*{{cite journal|author=Kungl. Vetenskapsakademien|year=2003|title=Time-series econometrics: Cointegration and autoregressive conditional heteroskedasticity|journal=Advanced Information on the Bank of Sweden Prize in Economic Sciences in Memory of Alfred Nobel|publisher=The Royal Swedish Academy of Sciences}}
*{{cite journal|last1=Arianos|first1=S.|last2=Carbone|first2=A.|last3=Turk|first3=C.|year=2011|title=Self-similarity of high-order moving averages|journal=[[Physical Review E]]|volume=84|issue=4|pages=046113|doi=10.1103/physreve.84.046113|pmid=22181233|bibcode=2011PhRvE..84d6113A |url=http://porto.polito.it/2488907/}}
*{{cite book|last=Altman|first=D.G.|year=1991|title=Practical Statistics for Medical Research|publisher=Chapman and Hall|location=London|isbn=041227630-5|pages=212-220}}

{{Statistics|analysis}}

[[Category:Regression with time series structure]]
[[Category:Statistical forecasting]]
[[Category:Change detection]]