Editing Alpha compositing

{{short description|Operation in computer graphics}}

[[Image:Hue alpha falloff.svg|thumb|A [[Visible spectrum|color spectrum]] image with an alpha channel that falls off to zero at its base, where it is blended with the background color.]]

In [[computer graphics]], '''alpha compositing''' or '''alpha blending''' is the process of combining one image with a background to create the appearance of partial or full [[Transparency (graphic)|transparency]].<ref>{{Cite web|title=Definition of alpha blending|url=https://www.pcmag.com/encyclopedia/term/alpha-blending|access-date=2021-08-07|website=PCMAG|language=en}}</ref> It is often useful to render [[picture element]]s (pixels) in separate passes or layers and then combine the resulting [[2D computer graphics|2D images]] into a single, final image called the [[compositing|composite]]. Compositing is used extensively in [[film]] when combining [[Computer-generated imagery|computer-rendered image]] elements with [[live footage]]. Alpha blending is also used in 2D computer graphics to put [[rasterized]] foreground elements over a background.

In order to combine the picture elements of the images correctly, it is necessary to keep an associated ''[[matte (filmmaking)|matte]]'' for each element in addition to its color. This matte layer contains the coverage information—the shape of the geometry being drawn—making it possible to distinguish between parts of the image where something was drawn and parts that are empty.

Although the most basic operation of combining two images is to put one over the other, there are many operations, or [[blend modes]], that are used.

== History ==
The concept of an alpha channel was introduced by [[Alvy Ray Smith]] and [[Ed Catmull]] in the late 1970s at the [[New York Institute of Technology Computer Graphics Lab]]. Bruce A. Wallace derived the same straight '''over''' operator based on a physical [[reflectance]]/[[transmittance]] model in 1981.<ref name=Wallace>{{cite conference |last=Wallace |first=Bruce A. |book-title=Proceedings of the 8th annual conference on Computer graphics and interactive techniques - SIGGRAPH '81 |title=Merging and transformation of raster images for cartoon animation |date=1981 |location=New York City, New York |publisher=ACM Press |volume=15 |issue=3 |pages=[https://archive.org/details/siggraph81confer15sigg/page/253 253–262] |citeseerx=10.1.1.141.7875 |doi=10.1145/800224.806813 |isbn=0-89791-045-1 |s2cid=1147910 |url-access=registration |url=https://archive.org/details/siggraph81confer15sigg/page/253 |doi-access=free }}</ref> A 1984 paper by [[Thomas Porter (Pixar)|Thomas Porter]] and [[Tom Duff]] introduced premultiplied alpha using a geometrical approach.<ref name=Duff>{{Cite conference|last1=Porter|first1=Thomas|author-link=Thomas Porter (Pixar)|last2=Duff|first2=Tom|author-link2=Tom Duff|book-title=Proceedings of the 11th annual conference on Computer graphics and interactive techniques - SIGGRAPH '84|title=Compositing digital images|date=July 1984|url=http://graphics.pixar.com/library/Compositing/paper.pdf|url-status=live |language=en|location=New York City, New York|publisher=ACM Press|volume=18|issue=3|pages=253–259|doi=10.1145/800031.808606|isbn=9780897911382|s2cid=18663039|archive-url=https://web.archive.org/web/20110429041428/http://graphics.pixar.com/library/Compositing/paper.pdf|archive-date=2011-04-29|access-date=2019-03-11}}</ref>

The use of the term ''alpha'' is explained by Smith as follows: "We called it that because of the [[Convex combination|classic linear interpolation formula]] <math>\alpha A + (1-\alpha)B</math> that uses the Greek letter <math>\alpha</math> (alpha) to control the amount of interpolation between, in this case, two images A and B".<ref>{{cite web|author=Alvy Ray Smith|date=1995-08-15|title=Alpha and the History of Digital Compositing|url=http://alvyray.com/Memos/CG/Microsoft/7_alpha.pdf|work=alvyray.com|page=6|archiveurl=https://web.archive.org/web/20211025162251/http://alvyray.com/Memos/CG/Microsoft/7_alpha.pdf|archivedate=2021-10-25}}</ref> That is, when compositing image A atop image B, the value of <math>\alpha</math> in the formula is taken directly from A's alpha channel.

==Description==
In a 2D image a color combination is stored for each picture element (pixel), often a combination of red, green and blue ([[RGB color model|RGB]]). When alpha compositing is in use, each pixel has an additional numeric value stored in its '''alpha channel''', with a value ranging from 0 to 1. A value of 0 means that the pixel is fully [[transparency and translucency|transparent]] and the color in the pixel beneath will show through. A value of 1 means that the pixel is fully opaque.

With the existence of an alpha channel, it is possible to express compositing image operations using a ''compositing algebra''. For example, given two images ''A'' and ''B'', the most common compositing operation is to combine the images so that ''A'' appears in the foreground and ''B'' appears in the background. This can be expressed as ''A'' '''over''' ''B''. In addition to '''over''', Porter and Duff<ref name=Duff/> defined the compositing operators '''in''', '''held out by''' (the phrase refers to [[Matte (filmmaking)#Garbage and holdout mattes|holdout matting]] and is usually abbreviated '''out'''), '''atop''', and '''xor''' (and the reverse operators '''rover''', '''rin''', '''rout''', and '''ratop''') from a consideration of choices in blending the colors of two pixels when their coverage is, conceptually, overlaid orthogonally:

[[Image:Alpha compositing.svg]]

As an example, the '''over''' operator can be accomplished by applying the following formula to each pixel:<ref name=Wallace/>
:<math>\begin{align}\alpha_o &= \phantom{~C_a}\alpha_a + \phantom{C_b}\alpha_b(1 - \alpha_a) \\
C_o &= \frac{ C_a \alpha_a + C_b \alpha_b (1 - \alpha_a) }{\alpha_o}
\end{align}</math>

Here <math>C_o</math>, <math>C_a</math> and <math>C_b</math> stand for the color components of the pixels in the result of the "over", image A, and image B respectively, applied to each color channel (red/green/blue) individually, whereas <math>\alpha_o</math>, <math>\alpha_a</math> and <math>\alpha_b</math> are the alpha values of the respective pixels.

The '''over''' operator is, in effect, the normal painting operation (see [[Painter's algorithm]]). The '''in''' and '''out''' operators are the alpha compositing equivalent of [[Clipping path|clipping]]. The two use only the alpha channel of the second image and ignore the color components. In addition, '''plus''' defines additive blending.<ref name=Duff/>

== Straight versus premultiplied ==
If an alpha channel is used in an image, there are two common representations that are available: straight (unassociated) alpha and premultiplied (associated) alpha.
* With '''straight alpha''', the RGB components represent the color of the object or pixel, disregarding its opacity. This is the method implied by the '''over''' operator in the previous section.
* With '''premultiplied alpha''', the RGB components represent the emission of the object or pixel, and the alpha represents the occlusion. The '''over''' operator then becomes:<ref name=Duff/>
*:<math>C_o = C_a + C_b(1 - \alpha_a)</math>
*:<math>\alpha_o = \alpha_a + \alpha_b(1 - \alpha_a)</math>

=== Comparison ===
The most significant advantage of premultiplied alpha is that it allows for correct blending, [[interpolation]], and [[Filter (signal processing)|filtering]]. Ordinary interpolation without premultiplied alpha leads to RGB information leaking out of fully transparent (A=0) regions, even though this RGB information is ideally invisible. When interpolating or filtering images with abrupt borders between transparent and opaque regions, this can result in borders of colors that were not visible in the original image. Errors also occur in areas of semitransparency because the RGB components are not correctly weighted, giving incorrectly high weighting to the color of the more transparent (lower alpha) pixels.<ref>{{cite web |title=Alpha Blending: To Pre or Not To Pre |url=https://developer.nvidia.com/content/alpha-blending-pre-or-not-pre |website=NVIDIA Developer |language=en |date=31 January 2013 |quote=However, something interesting happens when we generate the next mipmap level...}}</ref>

Premultiplied alpha may also be used to allow regions of regular alpha blending (e.g. smoke) and regions with [[Blend modes#Addition|additive blending mode]] (e.g. flame and glitter effects) to be encoded within the same image.<ref>{{cite web|url=https://tomforsyth1000.github.io/blog.wiki.html#%5B%5BPremultiplied%20alpha%5D%5D |website=TomF's Tech Blog - It's only pretending to be a wiki |access-date=8 May 2018 |url-status=live |archive-url=https://web.archive.org/web/20171212111056/http://tomforsyth1000.github.io/blog.wiki.html#%5B%5BPremultiplied+alpha%5D%5D |archive-date=12 December 2017 |title=TomF's Tech Blog - It's only pretending to be a wiki }}</ref><ref>{{cite web |last1=Trebilco |first1=Damian |title=To close to draw call (presentation on Pre-multiplied alpha) |url=https://github.com/dtrebilco/PreMulAlpha |website=GitHub |quote=By switching to pre-multiplied blend mode for all particle effects, and entire scene can be done with one draw call (assuming atlasing/2D array for the textures)... Another neat trick with pre-multiplied alpha is that if you have overlapping textures that are in known positions, you can pre-process them all down to one texture.}}</ref> This is represented by an RGBA triplet that express emission with no occlusion, such as (0.4, 0.3, 0.2, 0.0).

Another advantage of premultiplied alpha is performance; in certain situations, it can reduce the number of multiplication operations (e.g. if the image is used many times during later compositing). The Porter&ndash;Duff operations have a simple form only in premultiplied alpha.<ref name=Duff/> Some rendering pipelines expose a "straight alpha" API surface, but converts them into premultiplied alpha for performance.<ref>{{cite web |title=Premultiplied alpha |url=https://microsoft.github.io/Win2D/WinUI3/html/PremultipliedAlpha.htm |website=Win2D for WinUI3 |access-date=30 June 2023 |quote=Win2D uses straight alpha in its API surface, but premultiplied alpha for internal rendering operations.}}</ref>

One disadvantage of premultiplied alpha is that it can reduce the available relative precision in the RGB values when using integer or fixed-point representation for the color components. This may cause a noticeable loss of quality if the color information is later brightened or if the alpha channel is removed. In practice, this is not usually noticeable because during typical composition operations, such as OVER, the influence of the low-precision color information in low-alpha areas on the final output image (after composition) is correspondingly reduced. This loss of precision also makes premultiplied images easier to [[Data compression|compress]] using certain compression schemes, as they do not record the color variations hidden inside transparent regions, and can allocate fewer bits to encode low-alpha areas. The same “limitations” of lower quantisation bit depths such as 8 bit per channel are also present in imagery without alpha, and this argument is problematic as a result.

=== Examples ===
Assuming that the pixel color is expressed using ''straight'' (non-premultiplied) RGBA [[tuple]]s, a pixel value of (0, 0.7, 0, 0.5) implies a pixel that has 70% of the maximum green intensity and 50% opacity. If the color were fully green, its RGBA would be (0, 1, 0, 0.5). However, if this pixel uses premultiplied alpha, all of the RGB values (0, 0.7, 0) are multiplied, or scaled for occlusion, by the alpha value 0.5, which is appended to yield (0, 0.35, 0, 0.5). In this case, the 0.35 value for the G channel actually indicates 70% green emission intensity (with 50% occlusion). A pure green emission would be encoded as (0, 0.5, 0, 0.5). Knowing whether a file uses straight or premultiplied alpha is essential to correctly process or composite it, as a different calculation is required.

Emission with no occlusion cannot be represented in straight alpha. No conversion is available in this case.

== Image formats supporting alpha channels ==
The most popular image formats that support the alpha channel are [[Portable Network Graphics|PNG]] and [[Tagged Image File Format|TIFF]]. [[Graphics Interchange Format|GIF]] supports alpha channels, but is considered to be inefficient when it comes to file size. Support for alpha channels is present in some video codecs, such as Animation and Apple ProRes 4444 of the QuickTime format, or in the Techsmith multi-format codec.

The [[file format]] [[Windows bitmap|BMP]] generally does not support this channel; however, in different formats such as 32-bit (888–8) or 16-bit (444–4) it is possible to save the alpha channel, although not all systems or programs are able to read it: it is exploited mainly in some video games<ref>{{Cite web |title=Creating Textures |url=https://www.echos.ch/eod/help/texturen.htm |access-date=2023-05-25 |website=www.echos.ch}}</ref> or particular applications;<ref>{{Cite web |date=2016-03-04 |title=Extended Formats |url=http://home.arcor.de/zarfirbel/DXT%20BMP/DXTBmp.htm |access-date=2023-05-25 |archive-url=https://web.archive.org/web/20160304134021/http://home.arcor.de/zarfirbel/DXT%20BMP/DXTBmp.htm |archive-date=2016-03-04 }}</ref> specific programs have also been created for the creation of these BMPs.

{| class="wikitable sortable"
! File/Codec format<ref>{{cite web |url=https://pixelbakery.com/recipes/video-image-formats |title=List of Video/Image Formats Supporting Alpha Channels |last=Lambrecht |first=Jordan |date=2022-12-31 |access-date=2023-05-25 |website=Pixel Bakery Design Studio}}</ref> !! Maximum Depth 
!Type!! Browser support !! Media type !! Notes
|-
| [[Apple ProRes|Apple ProRes 4444]] || 16-bit 
| || None || Video (.mov) || ProRes is the successor of the [[Apple Intermediate Codec]]<ref>{{cite web|url=https://www.apple.com/au/finalcutstudio/finalcutpro/support.html|title=Final Cut Pro 6 - Broad Format Support.|website=apple.com|access-date=2024-08-13|archive-date=2011-06-08|archive-url=https://web.archive.org/web/20110608065728/http://www.apple.com/au/finalcutstudio/finalcutpro/support.html|url-status=dead}}</ref>
|-
|[[High Efficiency Video Coding|HEVC]] / h.265 ||10-bit 
| || Limited to [[Safari (web browser)|Safari]] ||Video (.hevc) || Intended successor to [[H.264]]<ref name=FirstJCTVCMeetingDresden>{{cite web|url=http://www.h265.net/2010/06/the-first-jct-vc-meeting-dresden-de.html |archive-url=https://web.archive.org/web/20100625022448/http://www.h265.net/2010/06/the-first-jct-vc-meeting-dresden-de.html |url-status=usurped |archive-date=June 25, 2010 |title=The First JCT-VC Meeting, Dresden, DE |author=Jie Dong |publisher=H265.net |date=2010-06-19 |access-date=2024-08-13}}</ref><ref name=StatusH265July2008>{{cite web|url=http://www.h265.net/2008/07/current-status-of-h265.html |archive-url=https://web.archive.org/web/20080712015245/http://www.h265.net/2008/07/current-status-of-h265.html |url-status=usurped |archive-date=July 12, 2008 |title=Current Status of H.265 (as at July 2008) |author=Jie Dong |publisher=H265.net |date=2008-07-01 |access-date=2024-08-13}}</ref><ref name=NGVCRequirements2009>{{cite web|url=http://www.h265.net/2009/04/the-preliminary-requirements-for-ngvc.html |archive-url=https://web.archive.org/web/20090513034217/http://www.h265.net/2009/04/the-preliminary-requirements-for-ngvc.html |url-status=usurped |archive-date=May 13, 2009 |title=The Preliminary Requirements for NGVC |author=Yu Liu |publisher=H265.net |date=2009-04-15 |access-date=2024-08-13}}</ref>
|-
|[[WebM]] (codec video VP8, VP9, or AV1) || 12-bit 
| || All modern browsers ||Video (.webm) || While VP8/VP9 is widely supported with modern browsers, AV1 still has limited support.<ref>{{Cite web |title=AV1 video format {{!}} Can I use... Support tables for HTML5, CSS3, etc |url=https://caniuse.com/av1 |access-date=2023-05-25 |website=caniuse.com}}</ref> Only [[Chromium (web browser)|Chromium]]-based browsers will display alpha layers.
|- 
|[[OpenEXR]] || 32-bit 
| || None || Image (.exr) || Has largest HDR spread.
|- 
| [[PNG]]|| 16-bit 
|straight|| All modern browsers || Image (.png) ||
|-
| [[APNG]]|| 24-bit 
|straight|| Moderate support || Image (.apng) || Supports animation.<ref>{{Cite web |title=Digital Audio Broadcasting (DAB); MOT SlideShow; User Application (pdf) Specification |url=http://www.etsi.org/deliver/etsi_ts/101400_101499/101499/02.02.01_60/ts_101499v020201p.pdf |access-date=13 August 2024 |publisher=ETSI}}</ref>
|- 
| [[TIFF]]|| 32-bit 
|both|| None || Image (.tiff) ||
|-
|[[GIF]]|| 8-bit 
| || All modern browsers || Image (.gif) || Browsers generally do not support GIF alpha layers.
|-
|[[SVG]] || 32-bit 
|straight|| All modern browsers || Image (.svg) || Based on CSS color.<ref name="colr">{{cite web |url=http://www.w3.org/TR/SVG11/color.html |title=SVG specification, "Color" |publisher=World Wide Web Consortium |date=14 January 2003 |access-date=13 August 2024 |archive-date=7 September 2009 |archive-url=https://web.archive.org/web/20090907075028/http://www.w3.org/TR/SVG11/color.html |url-status=live }}</ref>
|-
|[[JPEG XL]]
|32-bit
|both
|Moderate support
|Image (.jxl)
|Allows lossy and HDR.<ref>{{cite web |last=@chromium.org |first=de... |date=2022-08-24 |title=JPEG XL decoding support (image/jxl) in blink (tracking bug) |url=https://bugs.chromium.org/p/chromium/issues/detail?id=1178058 |access-date=2024-08-13 |website=bugs.chromium.org}}</ref>
|}

== Gamma correction ==
[[File:Mix lazy.png|thumb|Alpha blending, not taking into account gamma correction]]

[[File:Mix precise.png|thumb|Alpha blending, taking<br> into account gamma correction]]

The RGB values of typical digital images do not directly correspond to the physical light intensities, but are rather compressed by a [[gamma correction]] function:

: <math>C_\text{encoded} = C_\text{linear}^{1/\gamma}</math>

This transformation better utilizes the limited number of bits in the encoded image by choosing <math>\gamma</math> that better matches the non-linear human perception of luminance.

Accordingly, computer programs that deal with such images must decode the RGB values into a linear space (by undoing the gamma-compression), blend the linear light intensities, and re-apply the gamma compression to the result:<ref>{{cite web|url=https://www.youtube.com/watch?v=LKnqECcg6Gw| archive-url=https://ghostarchive.org/varchive/youtube/20211122/LKnqECcg6Gw| archive-date=2021-11-22 | url-status=live|title=Computer Color is Broken|author=Minute Physics|date=March 20, 2015|website=[[YouTube]]}}{{cbignore}}</ref><ref>{{cite web |title=What every coder should know about gamma |url=https://blog.johnnovak.net/2016/09/21/what-every-coder-should-know-about-gamma/ |last=Novak|first=John|date=September 21, 2016}}</ref>{{Failed verification|date=October 2022}}

:''<math id="C_o = \left(\frac{  C_a^\gamma \alpha_a + C_b^\gamma \alpha_b (1 - \alpha_a)  }{\alpha_o}\right)^{1/…" qid="Q281055">C_o = \left(\frac{  C_a^\gamma \alpha_a + C_b^\gamma \alpha_b (1 - \alpha_a)  }{\alpha_o}\right)^{1/\gamma}</math>''

When combined with premultiplied alpha, pre-multiplication is done in linear space, prior to gamma compression.<ref>{{cite web |title=Gamma Correction vs. Premultiplied Pixels – Søren Sandmann Pedersen |url=http://ssp.impulsetrain.com/gamma-premult.html |website=ssp.impulsetrain.com}}</ref> This results in the following formula:

:<math>C_o = \left( C_a^\gamma + C_b^\gamma (1 - \alpha_a) \right)^{1/\gamma}</math>

Note that the alpha channel may or may not undergo gamma-correction, even when the color channels do.

==Other transparency methods==
Although used for similar purposes, [[Palette (computing)#Transparent color in palettes|transparent color]]s and [[Mask (computing)#Image masks|image mask]]s do not permit the smooth blending of the superimposed image pixels with those of the background (only whole image pixels or whole background pixels allowed).

A similar effect can be achieved with a 1-bit alpha channel, as found in the 16-bit RGBA [[high color]] mode of the [[Truevision TGA]] [[image file format]] and related [[TARGA]] and AT-Vista/NU-Vista display adapters' high color graphic mode. This mode devotes 5 bits for every primary RGB color ([[List of monochrome and RGB color formats#15-bit RGB|15-bit RGB]]) plus a remaining bit as the "alpha channel".

[[Dither]]ing can be used to simulate partial occlusion where only 1-bit alpha is available.

For some applications, a single alpha channel is not sufficient: a [[Stained glass|stained-glass window]], for instance, requires a separate transparency channel for each RGB channel to model the red, green and blue transparency separately. More alpha channels can be added for accurate spectral color filtration applications.

Some [[order-independent transparency]] methods replace the '''over''' operator with a commutative approximation.<ref>{{cite journal |last1=McGuire |first1=Morgan |last2=Bavoil |first2=Louis |date=2013 |title=Weighted Blended Order-Independent Transparency |journal=Journal of Computer Graphics Techniques|volume=2 |issue=2 |pages=122–141 |url=https://www.jcgt.org/published/0002/02/09/ }}</ref>

==See also==
{{div col |colwidth=12em |content=
*[[Alpha to coverage]]
*[[Bit blit]]
*[[Blend modes]]
*[[Digital compositing]]
*[[Mask (computing)#Image masks|Image masks]]
*[[Magic Pink]]
*[[Portable Network Graphics]]
*[[RGBA color space]]
*[[Texture splatting]]
*[[Transparency (graphic)]]
*[[Palette (computing)#Transparent color in palettes|Transparent color in palettes]]
*[[HTML video#Transparent video|Transparent videos in web browsers]]
*[[Truevision TGA]]

}} <!-- end "content=" -->

==References==
{{reflist|25em}}

==External links==
* [http://keithp.com/~keithp/porterduff/p253-porter.pdf Compositing Digital Images - Thomas Porter and Tom Duff (Original Paper)]
* [http://www.cs.princeton.edu/courses/archive/fall00/cs426/papers/smith95a.pdf Image Compositing Fundamentals]
* [https://web.archive.org/web/20180726161540/http://www.svgopen.org/2005/papers/abstractsvgopen/ Understand Compositing and Color extensions in SVG 1.2 in 30 minutes!]
* [http://dvd-hq.info/alpha_matting.php Alpha Matting and Premultiplication]

{{Computer graphics}}

{{DEFAULTSORT:Alpha Compositing}}
[[Category:Computer graphics algorithms]]