Editing Student's t-test (section)

==Related statistical tests==
===Alternatives to the ''t''-test for location problems===
The ''t''-test provides an exact test for the equality of the means of two i.i.d. normal populations with unknown, but equal, variances. ([[Welch's t-test|Welch's ''t''-test]] is a nearly exact test for the case where the data are normal but the variances may differ.)  For moderately large samples and a one tailed test, the ''t''-test is relatively robust to moderate violations of the normality assumption.<ref name="Sawilowsky-Blair">{{cite journal |last1=Sawilowsky |first1=Shlomo S. |last2=Blair |first2=R. Clifford |year=1992 |title=A More Realistic Look at the Robustness and Type II Error Properties of the ''t'' Test to Departures From Population Normality |journal=Psychological Bulletin |volume=111 |issue=2 |pages=352–360 |doi=10.1037/0033-2909.111.2.352}}</ref> In large enough samples, the ''t''-test asymptotically approaches the [[Z-test|''z''-test]], and becomes robust even to large deviations from normality.<ref name=":0" />

If the data are substantially non-normal and the sample size is small, the ''t''-test can give misleading results. See [[location testing for Gaussian scale mixture distributions|Location test for Gaussian scale mixture distributions]] for some theory related to one particular family of non-normal distributions.

When the normality assumption does not hold, a [[non-parametric]] alternative to the ''t''-test may have better [[statistical power]]. However, when data are non-normal with differing variances between groups, a ''t''-test may have better [[Type 1 error|type-1 error]] control than some non-parametric alternatives.<ref>{{Cite journal|last=Zimmerman|first=Donald W.|date=January 1998|title=Invalidation of Parametric and Nonparametric Statistical Tests by Concurrent Violation of Two Assumptions|journal=The Journal of Experimental Education|volume=67|issue=1|pages=55–68|doi=10.1080/00220979809598344|issn=0022-0973}}</ref> Furthermore, non-parametric methods, such as the [[Mann–Whitney U test|Mann-Whitney U test]] discussed below, typically do not test for a difference of means, so should be used carefully if a difference of means is of primary scientific interest.<ref name=":0"/> For example, Mann-Whitney U test will keep the type 1 error at the desired level alpha if both groups have the same distribution. It will also have power in detecting an alternative by which group B has the same distribution as A but after some shift by a constant (in which case there would indeed be a difference in the means of the two groups). However, there could be cases where group A and B will have different distributions but with the same means (such as two distributions, one with positive skewness and the other with a negative one, but shifted so to have the same means). In such cases, MW could have more than alpha level power in rejecting the Null hypothesis but attributing the interpretation of difference in means to such a result would be incorrect.

In the presence of an [[outlier]], the ''t''-test is not robust. For example, for two independent samples when the data distributions are asymmetric  (that is, the distributions are [[skewness|skewed]]) or the distributions have large tails, then the Wilcoxon rank-sum test (also known as the [[Mann–Whitney U test|Mann–Whitney ''U'' test]]) can have three to four times higher power than the ''t''-test.<ref name="Sawilowsky-Blair"/><ref>{{cite journal |last1=Blair |first1=R. Clifford |last2=Higgins |first2=James J. |journal=Journal of Educational Statistics |year=1980 |pages=309–335 | volume=5 |issue=4 |title=A Comparison of the Power of Wilcoxon's Rank-Sum Statistic to That of Student's ''t'' Statistic Under Various Nonnormal Distributions |doi=10.2307/1164905 |jstor=1164905}}</ref><ref>{{cite journal |last1=Fay |first1=Michael P. |last2=Proschan |first2=Michael A. |journal=Statistics Surveys |year=2010 |pages=1–39 |volume=4 |url=http://www.i-journals.org/ss/viewarticle.php?id=51 |title=Wilcoxon–Mann–Whitney or ''t''-test? On assumptions for hypothesis tests and multiple interpretations of decision rules |doi=10.1214/09-SS051 |pmid=20414472 |pmc=2857732}}</ref> The nonparametric counterpart to the paired samples ''t''-test is the [[Wilcoxon signed-rank test]] for paired samples. For a discussion on choosing between the ''t''-test and nonparametric alternatives, see Lumley, et al. (2002).<ref name=":0" />

One-way [[analysis of variance]] (ANOVA) generalizes the two-sample ''t''-test when the data belong to more than two groups.

===A design which includes both paired observations and independent observations===
When both paired observations and independent observations are present in the two sample design, assuming  data are missing completely at random (MCAR), the paired observations or independent observations may be discarded in order to proceed with the standard tests above. Alternatively making use of all of the available data, assuming normality and MCAR, the generalized partially overlapping samples ''t''-test could be used.<ref name="Partover">{{cite journal|last1=Derrick|first1=B|last2=Toher|first2=D|last3=White|first3=P|title=How to compare the means of two samples that include paired observations and independent observations: A companion to Derrick, Russ, Toher and White (2017)|journal=The Quantitative Methods for Psychology|date=2017|volume=13|issue=2|pages=120–126|doi=10.20982/tqmp.13.2.p120|url=http://eprints.uwe.ac.uk/31765/1/How%20to%20compare%20means......%20BD_DT_PW.pdf|doi-access=free}}</ref>

===Multivariate testing===
{{main|Hotelling's T-squared distribution}}
A generalization of Student's ''t'' statistic, called [[Hotelling's t-squared statistic|Hotelling's ''t''-squared statistic]], allows for the testing of hypotheses on multiple (often correlated) measures within the same sample. For instance, a researcher might submit a number of subjects to a personality test consisting of multiple personality scales (e.g. the [[Minnesota Multiphasic Personality Inventory]]). Because measures of this type are usually positively correlated, it is not advisable to conduct separate univariate ''t''-tests to test hypotheses, as these would neglect the covariance among measures and inflate the chance of falsely rejecting at least one hypothesis ([[Type I error]]). In this case a single multivariate test is preferable for hypothesis testing. [[Fisher's Method#Limitations of independent assumption|Fisher's Method]] for combining multiple tests with ''[[Type I and type II errors#Type I error|alpha]]'' reduced for positive correlation among tests is one. Another is Hotelling's ''T''{{isup|2}} statistic follows a ''T''{{isup|2}} distribution. However, in practice the distribution is rarely used, since tabulated values for ''T''{{isup|2}} are hard to find. Usually, ''T''{{isup|2}} is converted instead to an ''F'' statistic.

For a one-sample multivariate test, the hypothesis is that the mean vector ({{math|'''μ'''}}) is equal to a given vector ({{math|'''μ'''<sub>0</sub>}}). The test statistic is [[Hotelling's t-squared statistic|Hotelling's ''t''{{isup|2}}]]:

:<math>
t^2=n(\bar{\mathbf x}-{\boldsymbol\mu_0})'{\mathbf S}^{-1}(\bar{\mathbf x}-{\boldsymbol\mu_0})
</math>

where {{math|''n''}} is the sample size, {{math|{{overline|'''x'''}}}} is the vector of column means and {{math|'''S'''}} is an {{math|''m'' × ''m''}} [[sample covariance matrix]].

For a two-sample multivariate test, the hypothesis is that the mean vectors ({{math|'''μ'''<sub>1</sub>, '''μ'''<sub>2</sub>}}) of two samples are equal. The test statistic is [[Hotelling's two-sample t-squared statistic|Hotelling's two-sample ''t''{{isup|2}}]]:

:<math>t^2 = \frac{n_1 n_2}{n_1+n_2}\left(\bar{\mathbf x}_1-\bar{\mathbf x}_2\right)'{\mathbf S_\text{pooled}}^{-1}\left(\bar{\mathbf x}_1-\bar{\mathbf x}_2\right).</math>

=== The two-sample ''t''-test is a special case of simple linear regression ===
{{unsourced section|date=March 2025}}

The two-sample ''t''-test is a special case of simple [[linear regression]] as illustrated by the following example.

A clinical trial examines 6 patients given drug or placebo. Three (3) patients get 0 units of drug (the placebo group). Three (3) patients get 1 unit of drug (the active treatment group). At the end of treatment, the researchers measure the change from baseline in the number of words that each patient can recall in a memory test.

[[File:Graph_of_word_recall_vs_drug_dose.svg|300px|alt=Scatter plot with six point. Three points on the left and are aligned vertically at the drug dose of 0 units. And the other three points on the right and are aligned vertically at the drug dose of 1 unit.]]

A table of the patients' word recall and drug dose values are shown below. 

{| {{Table}}
! Patient !! drug.dose !! word.recall
|-
! 1
! 0
| 1
|-
! 2
! 0
| 2
|-
! 3
! 0
| 3
|-
! 4
! 1
| 5
|-
! 5
! 1
| 6
|-
! 6
! 1
| 7
|}

Data and code are given for the analysis using the [[R programming language]] with the <code>t.test</code> and <code>lm</code>functions for the t-test and linear regression. Here are the same (fictitious) data above generated in R.

<syntaxhighlight lang="R">
> word.recall.data=data.frame(drug.dose=c(0,0,0,1,1,1), word.recall=c(1,2,3,5,6,7)) 
</syntaxhighlight>

Perform the ''t''-test. Notice that the assumption of equal variance, <code>var.equal=T</code>, is required to make the analysis exactly equivalent to simple linear regression.

<syntaxhighlight lang="R">
> with(word.recall.data, t.test(word.recall~drug.dose, var.equal=T))
</syntaxhighlight>

Running the R code gives the following results.
*The mean word.recall in the 0 drug.dose group is 2. 
*The mean word.recall in the 1 drug.dose group is 6. 
*The difference between treatment groups in the mean word.recall is 6 – 2 = 4.
* The difference in word.recall between drug doses is significant (p=0.00805).

Perform a linear regression of the same data. Calculations may be performed using the R function <code>lm()</code> for a linear model. 
<syntaxhighlight lang="R">
> word.recall.data.lm =  lm(word.recall~drug.dose, data=word.recall.data)
> summary(word.recall.data.lm)
</syntaxhighlight>

The linear regression provides a table of coefficients and p-values.

{| {{Table}}
! Coefficient !! Estimate!! Std. Error !! t value !! P-value 
|-
! Intercept
! 2
! 0.5774
! 3.464
| 0.02572
|-
! drug.dose
! 4
! 0.8165
! 4.899
| 0.000805
|}

The table of coefficients gives the following results.
*The estimate value of 2 for the intercept is the mean value of the word recall when the drug dose is 0. 
*The estimate value of 4 for the drug dose indicates that for a 1-unit change in drug dose (from 0 to 1) there is a 4-unit change in mean word recall (from 2 to 6). This is the slope of the line joining the two group means.
*The p-value that the slope of 4 is different from 0 is p = 0.00805.

The coefficients for the linear regression specify the slope and intercept of the line that joins the two group means, as illustrated in the graph. The intercept is 2 and the slope is 4.

[[File:Regression_lines_with_slopes_4_and_0.jpg|400px|Regression lines]]

Compare the result from the linear regression to the result from the ''t''-test.
* From the ''t''-test, the difference between the group means is 6-2=4. 
*From the regression, the slope is also 4 indicating that a 1-unit change in drug dose (from 0 to 1) gives a 4-unit change in mean word recall (from 2 to 6).
* The ''t''-test ''p''-value for the difference in means, and the regression p-value for the slope, are both 0.00805. The methods give identical results.

This example shows that, for the special case of a simple linear regression where there is a single x-variable that has values 0 and 1, the ''t''-test gives the same results as the linear regression. The relationship can also be shown algebraically.

Recognizing this relationship between the ''t''-test and linear regression facilitates the use of multiple linear regression and multi-way [[analysis of variance]]. These alternatives to ''t''-tests allow for the inclusion of additional [[Dependent and independent variables|explanatory variables]]  that are associated with the response. Including such additional explanatory variables using regression or anova reduces the otherwise unexplained [[variance]], and commonly yields greater [[Power of a test|power]] to detect differences than do two-sample ''t''-tests.