Editing Light field (section)

== Image refocusing ==
Because light field provides spatial and angular information, we can alter the position of focal planes after exposure, which is often termed ''refocusing''. The principle of refocusing is to obtain conventional 2-D photographs from a light field through the integral transform. The transform takes a lightfield as its input and generates a photograph focused on a specific plane.

Assuming <math>L_{F}(s,t,u,v)</math> represents a 4-D light field that records light rays traveling from position <math>(u,v)</math> on the first plane to position <math>(s,t)</math> on the second plane, where <math>F</math> is the distance between two planes, a 2-D photograph at any depth <math>\alpha F</math> can be obtained from the following integral transform:<ref name="renng">{{Cite book|last=Ng|first=Ren|title=ACM SIGGRAPH 2005 Papers |chapter=Fourier slice photography |date=2005|chapter-url=http://dx.doi.org/10.1145/1186822.1073256|pages=735–744 |location=New York, New York, USA|publisher=ACM Press|doi=10.1145/1186822.1073256|isbn=9781450378253 |s2cid=1806641 }}</ref>

:<math>
  \mathcal{P}_{\alpha}\left[L_{F}\right](s, t) =
  {1 \over \alpha^2 F^2}\iint L_F\left(u\left(1 - \frac{1}{\alpha}\right) + \frac{s}{\alpha}, v\left(1 - \frac{1}{\alpha}\right) + \frac{t}{\alpha}, u, v\right)~dudv
</math>,

or more concisely,

:<math>\mathcal{P}_{\alpha}\left[L_{F}\right](\boldsymbol{s})=\frac{1}{\alpha^{2} F^{2}} \int L_{F}\left(\boldsymbol{u}\left(1-\frac{1}{\alpha}\right)+\frac{\boldsymbol{s}}{\alpha}, \boldsymbol{u}\right) d \boldsymbol{u}</math>,

where <math>\boldsymbol{s}=(s,t)</math>, <math>\boldsymbol{u}=(u,v)</math>, and <math>\mathcal{P}_{\alpha}\left[\cdot\right]</math> is the photography operator.

In practice, this formula cannot be directly used because a plenoptic camera usually captures discrete samples of the lightfield <math>L_{F}(s,t,u,v)</math>, and hence resampling (or interpolation) is needed to compute <math display="inline"> L_{F}\left(\boldsymbol{u}\left(1-\frac{1}{\alpha}\right)+\frac{\boldsymbol{s}}{\alpha}, \boldsymbol{u}\right)</math>. Another problem is high computational complexity. To compute an <math>N\times N</math> 2-D photograph from an <math>N\times N\times N\times N</math> 4-D light field, the complexity of the formula is <math>O(N^4)</math>.<ref name="renng" />

=== Fourier slice photography ===
One way to reduce the complexity of computation is to adopt the concept of [[Projection-slice theorem|Fourier slice theorem]]:<ref name="renng" /> The photography operator <math>\mathcal{P}_{\alpha}\left[\cdot\right]</math> can be viewed as a shear followed by projection. The result should be proportional to a dilated 2-D slice of the 4-D Fourier transform of a light field. More precisely, a refocused image can be generated from the [[Light field microscopy|4-D Fourier spectrum]] of a light field by extracting a 2-D slice, applying an inverse 2-D transform, and scaling. The asymptotic complexity of the algorithm is <math>O(N^2 \operatorname{log}N)</math>.

=== Discrete focal stack transform ===
Another way to efficiently compute 2-D photographs is to adopt discrete focal stack transform (DFST).<ref>{{Cite journal|last1=Nava|first1=F. Pérez|last2=Marichal-Hernández|first2=J.G.|last3=Rodríguez-Ramos|first3=J.M.|date=August 2008|title=The Discrete Focal Stack Transform|url=https://ieeexplore.ieee.org/document/7080334|journal=2008 16th European Signal Processing Conference|pages=1–5}}</ref> DFST is designed to generate a collection of refocused 2-D photographs, or so-called [[Focus stacking|Focal Stack]]. This method can be implemeted by fast [[Fractional Fourier transform|fractional fourier transform]] (FrFT).

The discrete photography operator <math>\mathcal{P}_{\alpha}\left[\cdot\right]</math> is defined as follows for a lightfield <math>L_{F}(\boldsymbol {s},\boldsymbol {u})</math> sampled in a 4-D grid <math>\boldsymbol {s} = \Delta s \tilde{\boldsymbol {s}},</math> <math>\tilde{\boldsymbol {s}} = -\boldsymbol {n}_{\boldsymbol {s}}, ..., \boldsymbol {n}_{\boldsymbol {s}}</math>, <math>\boldsymbol {u} = \Delta u \tilde{\boldsymbol {u}}, \tilde{\boldsymbol {u}}=-\boldsymbol {n}_{\boldsymbol {u}},...,\boldsymbol {n}_{\boldsymbol {u}}</math>:

:<math>\mathcal{P}_{q}[L](\boldsymbol{s})=
\sum_{\tilde{\boldsymbol{u}}=-\boldsymbol{n}_{\boldsymbol{u}}}^{\boldsymbol{n}_{\boldsymbol{u}}} L(\boldsymbol{u} q+\boldsymbol{s}, \boldsymbol{u}) \Delta \boldsymbol{u}, 
\quad \Delta \boldsymbol{u}=\Delta u\Delta v,
\quad q=\left(1-\frac{1}{\alpha}\right)</math>

Because <math>(\boldsymbol{u} q+\boldsymbol{s}, \boldsymbol{u}) </math> is usually not on the 4-D grid, DFST adopts [[trigonometric interpolation]] to compute the non-grid values.

The algorithm consists of these steps:

* Sample the light field <math>L_{F}(\boldsymbol {s},\boldsymbol {u})</math> with the sampling period <math>\Delta s</math> and <math>\Delta u</math> and get the discretized light field <math>L^d_{F}(\boldsymbol {s},\boldsymbol {u})</math>.
* Pad <math>L^d_{F}(\boldsymbol {s},\boldsymbol {u})</math> with zeros such that the signal length is enough for FrFT without aliasing.
* For every <math>\boldsymbol {u}</math>, compute the [[Discrete Fourier transform]] of <math>L^d_{F}(\boldsymbol {s},\boldsymbol {u})</math>, and get the result <math>R1</math>.
* For every focal length <math>\alpha F</math>, compute the [[Fractional Fourier transform|fractional fourier transform]] of <math>R1</math>, where the order of the transform depends on <math>\alpha</math>, and get the result <math>R2</math>.
* Compute the inverse Discrete Fourier transform of <math>R2</math>.
* Remove the marginal pixels of <math>R2</math> so that each 2-D photograph has the size <math>(2{n}_{\boldsymbol {s}}+1)</math> by <math>(2{n}_{\boldsymbol {s}}+1)</math>