Editing Color balance (section)

==Mathematics of color balance==
Color balancing is sometimes performed on a three-component image (e.g., [[RGB color model|RGB]]) using a 3x3 [[matrix (mathematics)|matrix]]. This type of transformation is appropriate if the image was captured using the wrong white balance setting on a digital camera, or through a color filter. Changing the color balance of an image can improve classifier results on a trained ML model.

===Scaling monitor R, G, and B===
In principle, one wants to scale all relative luminances in an image so that objects which are believed to be [[grey|neutral]] appear so. If, say, a surface with <math>R=240</math> was believed to be a white object, and if 255 is the count which corresponds to white, one could multiply all [[red]] values by 255/240. Doing analogously for [[green]] and [[blue]] would result, at least in theory, in a color balanced image. In this type of transformation the 3x3 matrix is a [[diagonal matrix]].

: <math>\left[\begin{array}{c} R \\ G \\ B \end{array}\right]=\left[\begin{array}{ccc}255/R'_w & 0 & 0 \\ 0 & 255/G'_w & 0 \\ 0 & 0 & 255/B'_w\end{array}\right]\left[\begin{array}{c}R' \\ G' \\ B' \end{array}\right]</math>

where <math>R</math>, <math>G</math>, and <math>B</math> are the color balanced red, green, and blue components of a [[pixel]] in the image; <math>R'</math>, <math>G'</math>, and <math>B'</math> are the red, green, and blue components of the image before color balancing, and <math>R'_w</math>, <math>G'_w</math>, and <math>B'_w</math> are the red, green, and blue components of a pixel which is believed to be a white surface in the image before color balancing. This is a simple scaling of the red, green, and blue channels, and is why color balance tools in [[Photoshop]] have a white eyedropper tool.  It has been demonstrated that performing the white balancing in the phosphor set assumed by [[sRGB]] tends to produce large errors in chromatic colors, even though it can render the neutral surfaces perfectly neutral.<ref name="Viggiano2004">{{cite book|doi=10.1117/12.524922|chapter=Comparison of the accuracy of different white-balancing options as quantified by their color constancy|title=Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications V|volume=5301|pages=323–333|year=2004|last1=Viggiano|first1=J A Stephen|s2cid=8971750|editor3-first=Ricardo J|editor3-last=Motta|editor2-first=Nitin|editor2-last=Sampat|editor1-first=Morley M|editor1-last=Blouke}}</ref>

===Scaling X, Y, Z===
If the image may be transformed into [[CIE 1931 color space|CIE XYZ tristimulus values]], the color balancing may be performed there.  This has been termed a "wrong von Kries" transformation.<ref name=Terstiege1972>{{Cite journal
 | author = Heinz Terstiege
 | title = Chromatic adaptation: a state-of-the-art report
 | year = 1972
 | journal = Journal of Color Appearance
 | volume = 1
 | issue = 4
 | pages = 19–23 (cont. 40)
}}</ref><ref name="Fairchild1998">Mark D Fairchild, ''Color Appearance Models.'' Reading, MA: Addison-Wesley, 1998.</ref> Although it has been demonstrated to offer usually poorer results than balancing in monitor RGB, it is mentioned here as a bridge to other things. Mathematically, one computes:

:<math>\left[\begin{array}{c} X \\ Y \\ Z \end{array}\right]=\left[\begin{array}{ccc}X_w/X'_w & 0 & 0 \\ 0 & Y_w/Y'_w & 0 \\ 0 & 0 & Z_w/Z'_w\end{array}\right]\left[\begin{array}{c}X' \\ Y' \\ Z' \end{array}\right]</math>

where <math>X</math>, <math>Y</math>, and <math>Z</math> are the color-balanced tristimulus values; <math>X_w</math>, <math>Y_w</math>, and <math>Z_w</math> are the tristimulus values of the viewing illuminant (the white point to which the image is being transformed to conform to); <math>X'_w</math>, <math>Y'_w</math>, and <math>Z'_w</math> are the tristimulus values of an object believed to be white in the un-color-balanced image, and <math>X'</math>, <math>Y'</math>, and <math>Z'</math> are the tristimulus values of a pixel in the un-color-balanced image. If the tristimulus values of the monitor primaries are in a matrix <math>\mathbf{P}</math> so that:

:<math>\left[\begin{array}{c} X \\ Y \\ Z \end{array}\right]=\mathbf{P}\left[\begin{array}{c}L_R \\ L_G \\ L_B \end{array}\right]</math>

where <math>L_R</math>, <math>L_G</math>, and <math>L_B</math> are the un-[[gamma correction|gamma corrected]] monitor RGB, one may use:

:<math>\left[\begin{array}{c} L_R \\ L_G \\ L_B \end{array}\right]=\mathbf{P^{-1}}\left[\begin{array}{ccc}X_w/X'_w & 0 & 0 \\ 0 & Y_w/Y'_w & 0 \\ 0 & 0 & Z_w/Z'_w\end{array}\right]\mathbf{P}\left[\begin{array}{c}L_{R'} \\ L_{G'} \\ L_{B'} \end{array}\right]</math>

===Von Kries's method===
[[Johannes von Kries]], whose theory of [[rod cell|rods]] and three color-sensitive [[cone cell|cone]] types in the [[retina]] has survived as the dominant explanation of color sensation for over 100 years, motivated the method of converting color to the [[LMS color space]], representing the effective stimuli for the Long-, Medium-, and Short-wavelength cone types that are modeled as adapting independently.  A 3x3 matrix converts RGB or XYZ to LMS, and then the three LMS primary values are scaled to balance the neutral; the color can then be converted back to the desired final [[color space]]:<ref name=Sharma>{{Cite book| title = Digital Color Imaging Handbook | author =  Gaurav Sharma| url = https://books.google.com/books?id=AkByHKRGTsQC&q=%22von+Kries%22&pg=PA153 | publisher = [[CRC Press]] | year = 2003 | isbn = 978-0-8493-0900-7 | page=153 }}</ref>

:<math>\left[\begin{array}{c} L \\ M \\ S \end{array}\right]=\left[\begin{array}{ccc}1/L'_w & 0 & 0 \\ 0 & 1/M'_w & 0 \\ 0 & 0 & 1/S'_w\end{array}\right]\left[\begin{array}{c}L' \\ M' \\ S' \end{array}\right]</math>
where <math>L</math>, <math>M</math>, and <math>S</math> are the color-balanced LMS cone tristimulus values; <math>L'_w</math>, <math>M'_w</math>, and <math>S'_w</math> are the tristimulus values of an object believed to be white in the un-color-balanced image, and <math>L'</math>, <math>M'</math>, and <math>S'</math> are the tristimulus values of a pixel in the un-color-balanced image.

Matrices to convert to LMS space were not specified by von Kries, but can be derived from CIE color matching functions and LMS color matching functions when the latter are specified; matrices can also be found in reference books.<ref name=Sharma/>

===Scaling camera RGB===
By Viggiano's measure, and using his model of gaussian camera spectral sensitivities, most camera RGB spaces performed better than either monitor RGB or XYZ.<ref name="Viggiano2004"/>  If the camera's raw RGB values are known, one may use the 3x3 diagonal matrix:

: <math>\left[\begin{array}{c} R \\ G \\ B \end{array}\right]=\left[\begin{array}{ccc}255/R'_w & 0 & 0 \\ 0 & 255/G'_w & 0 \\ 0 & 0 & 255/B'_w\end{array}\right]\left[\begin{array}{c}R' \\ G' \\ B' \end{array}\right]</math>

and then convert to a working RGB space such as [[sRGB]] or [[Adobe RGB]] after balancing.
<!-- in progress!
However, if one has already converted to monitor RGB, one may still work in camera RGB if a 3x3 [[regular matrix]] <math>\mathbf{A}</math> characterizes the camera's color mixing behavior reasonably well, so that:

: <math>\left[\begin{array}{c} X \\ Y \\ Z \end{array}\right]\approx\mathbf{A}\left[\begin{array}{c}L_R \\ L_G \\ L_B \end{array}\right]</math>

(This matrix is included in the [[ICC profile]] for some cameras.<ref name="ICC01_2006">International Color Consortium, ''Specification ICC.1:2004-10 (Profile version 4.2.0.0) Image technology colour management — Architecture, profile format, and data structure'', (2006).</ref>) If this matrix is known, one computes:

:<math>\left[\begin{array}{c} L_R \\ L_G \\ L_B \end{array}\right]=\mathbf{P^{-1}A^{-1}}\left[\begin{array}{ccc}R_w/R'_w & 0 & 0 \\ 0 & G_w/G'_w & 0 \\ 0 & 0 & B_w/B'_w\end{array}\right]\mathbf{A\cdotP}\left[\begin{array}{c}L_{R'} \\ L_{G'} \\ L_{B'} \end{array}\right]</math>

where <math>\mathbf{P}</math> is the phosphor matrix mentioned in the previous section; <math>L_{Rw}</math, <math>G_w</math, <math>B_w</math are the
-->

===Preferred chromatic adaptation spaces===
Comparisons of images balanced by diagonal transforms in a number of different RGB spaces have identified several such spaces that work better than others, and better than camera or monitor spaces, for chromatic adaptation, as measured by several [[color appearance model]]s; the systems that performed statistically as well as the best on the majority of the image test sets used were the "Sharp", "Bradford", "CMCCAT", and "ROMM" spaces.<ref>{{Cite journal |url=http://infoscience.epfl.ch/getfile.py?recid=34049&mode=best |title=Chromatic Adaptation Performance of Different RGB Sensors |author1=Sabine Süsstrunk |author2=Jack Holm |author3=Graham D. Finlayson |editor-first1=Reiner |editor-first2=Gabriel G. |editor-last1=Eschbach |editor-last2=Marcu |journal=IS&T/SPIE Electronic Imaging |series=Color Imaging: Device-Independent Color, Color Hardcopy, and Graphic Arts VI |volume=4300 |date=January 2001 |pages=172–183 |access-date=2009-03-20 |archive-url=https://web.archive.org/web/20061018020916/http://infoscience.epfl.ch/getfile.py?mode=best&recid=34049 |archive-date=2006-10-18 |url-status=dead |doi=10.1117/12.410788 |s2cid=8140548 }}</ref>

===General illuminant adaptation===
The best color matrix for adapting to a change in illuminant is not necessarily a diagonal matrix in a fixed color space.  It has long been known that if the space of illuminants can be described as a linear model with ''N'' basis terms, the proper color transformation will be the weighted sum of ''N'' fixed linear transformations, not necessarily consistently diagonalizable.<ref>{{Cite book | author1 = Laurence T. Maloney | author2 = Brain A. Wandell | chapter = Color constancy: a method for recovering surface spectral reflectance | title = Readings in Computer Vision | editor1 = Martin A. Fischler | editor2 = Oscar Firschein | year = 1987 | publisher = Morgan-Kaufmann | isbn = 978-0-934613-33-0 | chapter-url = https://books.google.com/books?id=W5hLHUI8U-kC&q=maloney+wandell&pg=PA293 | url = https://archive.org/details/readingsincomput00fisc }}</ref>

=== Examples ===
{{Multiple image
| align = center
| direction = 
| width = 300
| footer = Comparison of resulted colors as shot by the digital camera for different light qualities (color temperature): Neutral, Warm and Cold.<ref name="photoskop">{{cite web |url=http://www.photoskop.com/player.html?l=wb&ch=0&sec=0 |title=photoskop: Interactive Photography Lessons |date=April 25, 2015}}</ref>
| image1 = Wb girl neutral.jpg
| caption1 = Neutral light
| image2 = Wb girl warm.jpg
| caption2 = Warm light
| image3 = Wb girl cold.jpg
| caption3 = Cold light
}}{{Multiple image
| align = center
| direction = 
| width = 300
| footer = Example of different white balance settings on digital camera for Neutral light.<ref name="photoskop"/>
| image1 = Wb girl neutral.jpg
| caption1 = Setting: As shot
| image2 = Wb girl cloudy.jpg
| caption2 = Setting: Cloudy
| image3 = Wb girl tungsten.jpg
| caption3 = Setting: Tungsten
}}