Editing Sensitivity analysis (section)

== Complementary research approaches for time-consuming simulations ==
A number of methods have been developed to overcome some of the constraints discussed above, which would otherwise make the estimation of sensitivity measures infeasible (most often due to [[computational expense]]). Generally, these methods focus on efficiently (by creating a metamodel of the costly function to be evaluated and/or by “ wisely ” sampling the factor space) calculating variance-based measures of sensitivity.

=== Metamodels ===
Metamodels (also known as emulators, surrogate models or response surfaces) are [[Data modeling|data-modeling]]/[[machine learning]] approaches that involve building a relatively simple mathematical function, known as an ''metamodels'', that approximates the input/output behavior of the model itself.<ref name="emcomp">{{cite journal | last1 = Storlie | first1 = C.B. | last2 = Swiler | first2 = L.P. | last3 = Helton | first3 = J.C. | last4 = Sallaberry | first4 = C.J. | year = 2009 | title = Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models | journal = Reliability Engineering & System Safety | volume = 94 | issue = 11| pages = 1735–1763 | doi=10.1016/j.ress.2009.05.007}}</ref> In other words, it is the concept of "modeling a model" (hence the name "metamodel"). The idea is that, although computer models may be a very complex series of equations that can take a long time to solve, they can always be regarded as a function of their inputs <math>Y=f(X)</math>. By running the model at a number of points in the input space, it may be possible to fit a much simpler metamodels <math>\hat{f}(X)</math>, such that <math>\hat{f}(X) \approx f(X)</math> to within an acceptable margin of error.<ref>{{Cite journal|last1=Wang|first1=Shangying|last2=Fan|first2=Kai|last3=Luo|first3=Nan|last4=Cao|first4=Yangxiaolu|last5=Wu|first5=Feilun|last6=Zhang|first6=Carolyn|last7=Heller|first7=Katherine A.|last8=You|first8=Lingchong|date=2019-09-25|title=Massive computational acceleration by using neural networks to emulate mechanism-based biological models|journal=Nature Communications|language=en|volume=10|issue=1|pages=4354|doi=10.1038/s41467-019-12342-y|issn=2041-1723|pmc=6761138|pmid=31554788|bibcode=2019NatCo..10.4354W}}</ref> Then, sensitivity measures can be calculated from the metamodel (either with Monte Carlo or analytically), which will have a negligible additional computational cost. Importantly, the number of model runs required to fit the metamodel can be orders of magnitude less than the number of runs required to directly estimate the sensitivity measures from the model.<ref name="oak">{{cite journal | last1 = Oakley | first1 = J. | last2 = O'Hagan | first2 = A. | year = 2004 | title = Probabilistic sensitivity analysis of complex models: a Bayesian approach | journal = J. R. Stat. Soc. B | volume = 66 | issue = 3| pages = 751–769 | doi=10.1111/j.1467-9868.2004.05304.x| citeseerx = 10.1.1.6.9720 | s2cid = 6130150 }}</ref>

Clearly, the crux of an metamodel approach is to find an <math>\hat{f}(X)</math> (metamodel) that is a sufficiently close approximation to the model <math>f(X)</math>. This requires the following steps,
# Sampling (running) the model at a number of points in its input space. This requires a sample design.
# Selecting a type of emulator (mathematical function) to use.
# "Training" the metamodel using the sample data from the model – this generally involves adjusting the metamodel parameters until the metamodel mimics the true model as well as possible.

Sampling the model can often be done with [[low-discrepancy sequences]], such as the [[Sobol sequence]] – due to mathematician [[Ilya M. Sobol]] or [[Latin hypercube sampling]], although random designs can also be used, at the loss of some efficiency. The selection of the metamodel type and the training are intrinsically linked since the training method will be dependent on the class of metamodel. Some types of metamodels that have been used successfully for sensitivity analysis include:

* [[Gaussian processes]]<ref name="oak" /> (also known as [[kriging]]), where any combination of output points is assumed to be distributed as a [[multivariate Gaussian distribution]]. Recently, "treed" Gaussian processes have been used to deal with [[Heteroscedasticity|heteroscedastic]] and discontinuous responses.<ref>{{cite journal |last1=Gramacy |first1=R. B. |last2=Taddy |first2=M. A. |title=Categorical Inputs, Sensitivity Analysis, Optimization and Importance Tempering with tgp Version 2, an R Package for Treed Gaussian Process Models |journal=Journal of Statistical Software |volume=33 |issue=6 |doi= 10.18637/jss.v033.i06 |url=https://cran.r-project.org/web/packages/tgp/vignettes/tgp2.pdf |year=2010 |doi-access=free }}</ref><ref>{{cite journal |last1=Becker |first1=W. |last2=Worden |first2=K. |last3=Rowson |first3=J. |title=Bayesian sensitivity analysis of bifurcating nonlinear models |journal=Mechanical Systems and Signal Processing |volume= 34|issue= 1–2|pages= 57–75|doi=10.1016/j.ymssp.2012.05.010 |bibcode=2013MSSP...34...57B |year=2013 |url=https://zenodo.org/record/890779 }}</ref>
* [[Random forest]]s,<ref name="emcomp" /> in which a large number of [[decision trees]] are trained, and the result averaged.
* [[Gradient boosting]],<ref name="emcomp" /> where a succession of simple regressions are used to weight data points to sequentially reduce error.
* [[polynomial chaos|Polynomial chaos expansions]],<ref>{{cite journal|last=Sudret|first=B.|date=2008|title=Global sensitivity analysis using polynomial chaos expansions|journal=Reliability Engineering & System Safety|volume=93|issue=7|pages=964–979|doi=10.1016/j.ress.2007.04.002}}</ref> which use [[orthogonal polynomials]] to approximate the response surface.
* [[Smoothing spline]]s,<ref>{{cite journal | last1 = Ratto | first1 = M. | last2 = Pagano | first2 = A. | year = 2010 | title = Using recursive algorithms for the efficient identification of smoothing spline ANOVA models | journal = AStA Advances in Statistical Analysis | volume = 94 | issue = 4| pages = 367–388 | doi=10.1007/s10182-010-0148-8| s2cid = 7678955 }}</ref> normally used in conjunction with [[high-dimensional model representation]] (HDMR) truncations (see below).
* Discrete [[Bayesian networks]],<ref>{{cite journal|last1= Cardenas |first1=IC|title= On the use of Bayesian networks as a meta-modeling approach to analyse uncertainties in slope stability analysis|journal =Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards|date=2019|volume=13|issue=1|pages=53–65|doi=10.1080/17499518.2018.1498524|bibcode=2019GAMRE..13...53C |s2cid=216590427 }}</ref> in conjunction with canonical models such as noisy models. Noisy models exploit information on the conditional independence between variables to significantly reduce dimensionality.

The use of an emulator introduces a [[machine learning]] problem, which can be difficult if the response of the model is highly [[nonlinear]]. In all cases, it is useful to check the accuracy of the emulator, for example using [[Cross-validation (statistics)|cross-validation]].

=== High-dimensional model representations (HDMR) ===
A [[high-dimensional model representation]] (HDMR)<ref>{{cite journal | last1 = Li | first1 = G. | last2 = Hu | first2 = J. | last3 = Wang | first3 = S.-W. | last4 = Georgopoulos | first4 = P. | last5 = Schoendorf | first5 = J. | last6 = Rabitz | first6 = H. | year = 2006 | title = Random Sampling-High Dimensional Model Representation (RS-HDMR) and orthogonality of its different order component functions | journal = Journal of Physical Chemistry A | volume = 110 | issue = 7| pages = 2474–2485 | doi=10.1021/jp054148m| pmid = 16480307 | bibcode = 2006JPCA..110.2474L }}</ref><ref>{{cite journal | last1 = Li | first1 = G. | year = 2002 | title = Practical approaches to construct RS-HDMR component functions | journal = Journal of Physical Chemistry | volume = 106 | issue = 37| pages = 8721–8733 | doi = 10.1021/jp014567t | bibcode = 2002JPCA..106.8721L }}</ref> (the term is due to H. Rabitz<ref>{{cite journal | last1 = Rabitz | first1 = H | year = 1989 | title = System analysis at molecular scale | journal = Science | volume = 246 | issue = 4927| pages = 221–226 | doi=10.1126/science.246.4927.221| pmid = 17839016 | bibcode = 1989Sci...246..221R| s2cid = 23088466 }}</ref>) is essentially an emulator approach, which involves decomposing the function output into a linear combination of input terms and interactions of increasing dimensionality. The HDMR approach exploits the fact that the model can usually be well-approximated by neglecting higher-order interactions (second or third-order and above). The terms in the truncated series can then each be approximated by e.g. polynomials or splines (REFS) and the response expressed as the sum of the main effects and interactions up to the truncation order. From this perspective, HDMRs can be seen as emulators which neglect high-order interactions; the advantage is that they are able to emulate models with higher dimensionality than full-order emulators.

=== Monte Carlo filtering ===
Sensitivity analysis via Monte Carlo filtering<ref>{{cite journal |last1=Hornberger |first1=G. |first2=R. |last2=Spear |year=1981 |title=An approach to the preliminary analysis of environmental systems |journal=Journal of Environmental Management |volume=7 |pages=7–18 }}</ref> is also a sampling-based approach, whose objective is to identify regions in the space of the input factors corresponding to particular values (e.g., high or low) of the output.