Editing Digital image processing (section)

== Digital image transformations ==

=== Filtering ===
Digital filters are used to blur and sharpen digital images. Filtering can be performed by:
* [[Kernel (image processing)#Convolution|convolution]] with specifically designed [[Kernel (image processing)|kernels]] (filter array) in the spatial domain<ref name=":0">{{Cite journal|last1=Zhang|first1=M. Z.|last2=Livingston|first2=A. R.|last3=Asari|first3=V. K.|date=2008|journal=International Journal of Computers and Applications|volume=30|issue=4|pages=298–308|doi=10.1080/1206212x.2008.11441909|title=A High Performance Architecture for Implementation of 2-D Convolution with Quadrant Symmetric Kernels|s2cid=57289814}}</ref>
* masking specific frequency regions in the frequency (Fourier) domain

The following examples show both methods:<ref name="Gonzalez 2008">{{cite book
  |last = Gonzalez
  |first = Rafael
  |title = Digital Image Processing, 3rd
  |publisher = Pearson Hall
  |date = 2008
  |isbn = 978-0-13-168728-8
}}</ref>

{| class="wikitable"
|-
! Filter type
! Kernel or mask
! Example
|-
| '''Original Image'''
| align="center" | <math>
\begin{bmatrix}
0 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 0
\end{bmatrix}
</math>
| [[File:Affine Transformation Original Checkerboard.jpg]]
|-
| '''[[lowpass|Spatial Lowpass]]'''
| align="center" | <math>
\frac{1}{9}\times
\begin{bmatrix}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{bmatrix}
</math>
| [[File:Spatial Mean Filter Checkerboard.png]]
|-
| '''[[Edge detection|Spatial Highpass]]'''
| align="center" | <math>
\begin{bmatrix}
0 & -1 & 0 \\
-1 & 4 & -1 \\
0 & -1 & 0
\end{bmatrix}
</math>
 | [[File:Spatial Laplacian Filter Checkerboard.png]]
|-
| '''[[Fast Fourier transform|Fourier Representation]]'''
| Pseudo-code:

image = checkerboard

F = Fourier Transform of image

Show Image: log(1+Absolute Value(F))

| align="center"| [[File:Fourier Space Checkerboard.png]]
|-
| '''Fourier Lowpass'''
| align="center"| [[File:Lowpass Butterworth Checkerboard.png]]
| align="center"| [[File:Lowpass FFT Filtered checkerboard.png]]
|-
| '''Fourier Highpass'''
| align="center"| [[File:Highpass Butterworth Checkerboard.png]]
| align="center"| [[File:Highpass FFT Filtered checkerboard.png]]
|-
|}

==== Image padding in Fourier domain filtering ====

Images are typically padded before being transformed to the Fourier space, the [[highpass filter]]ed images below illustrate the consequences of different padding techniques:

{| class="wikitable"
|-
! Zero padded
! Repeated edge padded
|-
| [[File:Highpass FFT Filtered checkerboard.png]]
| [[File:Highpass FFT Replicate.png]]
|-
|}

Notice that the highpass filter shows extra edges when zero padded compared to the repeated edge padding.

==== Filtering code examples ====

MATLAB example for spatial domain highpass filtering.

<syntaxhighlight lang="matlab">
img=checkerboard(20);                           % generate checkerboard
% **************************  SPATIAL DOMAIN  ***************************
klaplace=[0 -1 0; -1 5 -1;  0 -1 0];             % Laplacian filter kernel
X=conv2(img,klaplace);                          % convolve test img with
                                                % 3x3 Laplacian kernel
figure()
imshow(X,[])                                    % show Laplacian filtered
title('Laplacian Edge Detection')
</syntaxhighlight>

=== Affine transformations ===
[[Affine transformations]] enable basic image transformations including scale, rotate, translate, mirror and shear as is shown in the following examples:<ref name="Gonzalez 2008"/>

{| class="wikitable"
|-
! Transformation Name
! Affine Matrix
! Example
|-
| '''[[Identity operation|Identity]]'''
| align="center" | <math>
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
</math>
| [[File:Checkerboard identity.svg]]
|-
| '''[[Reflection (mathematics)|Reflection]]'''
| align="center" | <math>
\begin{bmatrix}
-1 & 0 & 0 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
</math>
| [[File:Checkerboard reflection.svg]]
|-
| '''[[Scale (ratio)|Scale]]'''
| align="center" | <math>
\begin{bmatrix}
c_x=2 & 0 & 0 \\
0 & c_y=1 & 0 \\
0 & 0 & 1
\end{bmatrix}
</math>
| [[File:Checkerboard scale.svg]]
|-
| '''[[Rotate]]'''
| align="center" | <math>
\begin{bmatrix}
\cos(\theta) & \sin(\theta) & 0 \\
-\sin(\theta) & \cos(\theta) & 0 \\
0 & 0 & 1
\end{bmatrix}
</math>
| [[File:Checkerboard rotate.svg]] where {{math|''θ'' {{=}} {{sfrac|π|6}} {{=}}30°}}
|-
| '''[[Shear matrix|Shear]]'''
| align="center" | <math>
\begin{bmatrix}
1 & c_x=0.5 & 0 \\
c_y=0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
</math>
| [[File:Checkerboard shear.svg]]
|-
|}
To apply the affine matrix to an image, the image is converted to matrix in which each entry corresponds to the pixel intensity at that location. Then each pixel's location can be represented as a vector indicating the coordinates of that pixel in the image, {{math|[''x'', ''y'']}}, where {{math|''x''}} and {{math|''y''}} are the row and column of a pixel in the image matrix. This allows the coordinate to be multiplied by an affine-transformation matrix, which gives the position that the pixel value will be copied to in the output image.

However, to allow transformations that require translation transformations, 3-dimensional [[homogeneous coordinates]] are needed. The third dimension is usually set to a non-zero constant, usually {{math|1}}, so that the new coordinate is {{math|[''x'', ''y'', 1]}}. This allows the coordinate vector to be multiplied by a 3×3 matrix, enabling translation shifts. Thus, the third dimension, i.e. the constant {{math|1}}, allows translation.

Because matrix multiplication is [[Associative property|associative]], multiple affine transformations can be combined into a single affine transformation by multiplying the matrix of each individual transformation in the order that the transformations are done. This results in a single matrix that, when applied to a point vector, gives the same result as all the individual transformations performed on the vector {{math|[''x'', ''y'', 1]}} in sequence. Thus a sequence of affine transformation matrices can be reduced to a single affine transformation matrix.

For example, 2-dimensional coordinates only permit rotation about the origin {{math|(0, 0)}}. But 3-dimensional homogeneous coordinates can be used to first translate any point to {{math|(0, 0)}}, then perform the rotation, and lastly translate the origin {{math|(0, 0)}} back to the original point (the opposite of the first translation). These three affine transformations can be combined into a single matrix—thus allowing rotation around any point in the image.<ref>{{Cite book|url=https://people.cs.clemson.edu/~dhouse/courses/401/notes/affines-matrices.pdf|title=Affine Transformations|last=House, Keyser|date=6 December 2016|website=Clemson|series=Foundations of Physically Based Modeling & Animation|publisher=A K Peters/CRC Press|isbn=978-1-4822-3460-2|access-date=26 March 2019|archive-url=https://web.archive.org/web/20170830052734/https://people.cs.clemson.edu/~dhouse/courses/401/notes/affines-matrices.pdf|archive-date=30 August 2017|url-status=live}}</ref>

=== Image denoising with mathematical morphology ===
[[Mathematical morphology]] (MM) is a nonlinear image processing framework that analyzes shapes within images by probing local pixel neighborhoods using a small, predefined function called a [[structuring element]]. In the context of grayscale images, MM is especially useful for denoising through [[Dilation (morphology)|dilation]] and [[Erosion_(morphology)|erosion]]—primitive operators that can be combined to build more complex filters.

Suppose we have:

* A discrete grayscale image: <math>
f =
\begin{bmatrix}
45 & 50 & 65 \\
40 & 60 & 55 \\
25 & 15 & 5
\end{bmatrix}, \quad f : \Omega \rightarrow \mathbb{R}, \quad \Omega = \{0, 1, 2\}^2,
</math>

* A structuring element: <math>
B =
\begin{bmatrix}
1 & 2 & 1 \\
2 & 1 & 1 \\
1 & 0 & 3
\end{bmatrix}, \quad B : \mathcal{S} \rightarrow \mathbb{R}, \quad \mathcal{S} = \{-1, 0, 1\}^2.
</math>
Here, <math>\mathcal{S}</math> defines the neighborhood of relative coordinates <math>(m, n)</math> over which local operations are computed. The values of <math>B(m, n)</math> bias the image during dilation and erosion.

; Dilation : Grayscale dilation is defined as:
<math display=block>
(f \oplus B)(i, j) = 
\max_{(m, n) \in \mathcal{S}} 
\Bigl\{
f(i+m, j+n) + B(m,n)
\Bigr\}.
</math>
:For example, the dilation at position {{math|(1, 1)}} is calculated as:

<math>
\begin{aligned}
(f \oplus B)(1,1) =
\max\!\Bigl(
&f(0,0)+B(-1,-1), &\;45+1;&\\
&f(1,0)+B( 0,-1), &\;50+2;&\\
&f(2,0)+B( 1,-1), &\;65+1;&\\
&f(0,1)+B(-1, 0), &\;40+2;&\\
&f(1,1)+B( 0, 0), &\;60+1;&\\
&f(2,1)+B( 1, 0), &\;55+1;&\\
&f(0,2)+B(-1, 1), &\;25+1;&\\
&f(1,2)+B( 0, 1), &\;15+0;&\\
&f(2,2)+B( 1, 1)  &\;5+3
\Bigr) = 66.
\end{aligned}
</math>

; Erosion : Grayscale erosion is defined as:
<math display=block>
(f \ominus B)(i,j) = 
\min_{(m,n) \in \mathcal{S}} 
\Bigl\{
f(i+m, j+n) - B(m,n)
\Bigr\}.
</math>
:For example, the erosion at position {{math|(1, 1)}} is calculated as:
<math>
\begin{aligned}
(f \ominus B)(1,1)=
\min\!\Bigl(
&f(0,0)-B(-1,-1), &\;45-1;&\\
&f(1,0)-B( 0,-1), &\;50-2;&\\
&f(2,0)-B( 1,-1), &\;65-1;&\\
&f(0,1)-B(-1, 0), &\;40-2;&\\
&f(1,1)-B( 0, 0), &\;60-1;&\\
&f(2,1)-B( 1, 0), &\;55-1;&\\
&f(0,2)-B(-1, 1), &\;25-1;&\\
&f(1,2)-B( 0, 1), &\;15-0;&\\
&f(2,2)-B( 1, 1)  &\;5-3
\Bigr)
=2.
\end{aligned}
</math>

==== Results ====

After applying dilation to <math>f</math>:
<math display=block>
\begin{bmatrix}
45 & 50 & 65 \\
40 & 66 & 55 \\
25 & 15 & 5
\end{bmatrix}
</math>

After applying erosion to <math>f</math>:
<math display=block>
\begin{bmatrix}
45 & 50 & 65 \\
40 & 2 & 55 \\
25 & 15 & 5
\end{bmatrix}
</math>

==== Opening and Closing ====

MM operations, such as [[Opening (morphology)|opening]] and [[Closing (morphology)|closing]], are composite processes that utilize both dilation and erosion to modify the structure of an image. These operations are particularly useful for tasks such as noise removal, shape smoothing, and object separation.

* ''Opening'': This operation is performed by applying erosion to an image first, followed by dilation. The purpose of opening is to remove small objects or noise from the foreground while preserving the overall structure of larger objects. It is especially effective in situations where noise appears as isolated bright pixels or small, disconnected features.

For example, applying opening to an image <math>f</math> with a structuring element <math>B</math> would first reduce small details (through erosion) and then restore the main shapes (through dilation). This ensures that unwanted noise is removed without significantly altering the size or shape of larger objects.

* ''Closing'': This operation is performed by applying dilation first, followed by erosion. Closing is typically used to fill small holes or gaps within objects and to connect broken parts of the foreground. It works by initially expanding the boundaries of objects (through dilation) and then refining the boundaries (through erosion).
For instance, applying closing to the same image <math>f</math> would fill in small gaps within objects, such as connecting breaks in thin lines or closing small holes, while ensuring that the surrounding areas are not significantly affected.

Both opening and closing can be visualized as ways of refining the structure of an image: opening simplifies and removes small, unnecessary details, while closing consolidates and connects objects to form more cohesive structures.

{| class="wikitable"
|-
! Structuring element
! Mask
! Code
! Example
|-
| '''Original Image'''
| None
| Use Matlab to read Original image
<syntaxhighlight lang="matlab">
original = imread('scene.jpg');
image = rgb2gray(original);
[r, c, channel] = size(image);
se = logical([1 1 1 ; 1 1 1 ; 1 1 1]);
[p, q] = size(se);
halfH = floor(p/2);
halfW = floor(q/2);
time = 3;           % denoising 3 times with all method
</syntaxhighlight>
| [[File:Lotus free.jpg|thumb|Original lotus]]
|-
|-
| '''[[dilation (morphology)|Dilation]]'''
| align="center" | <math>
\begin{bmatrix}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{bmatrix}
</math>
| Use Matlab to dilation
<syntaxhighlight lang="matlab">
imwrite(image, "scene_dil.jpg")
extractmax = zeros(size(image), class(image));
for i = 1 : time
    dil_image = imread('scene_dil.jpg');
    for col = (halfW + 1): (c - halfW)
        for row = (halfH + 1) : (r - halfH)
            dpointD = row - halfH;
            dpointU = row + halfH;
            dpointL = col - halfW;
            dpointR = col + halfW;
            dneighbor = dil_image(dpointD:dpointU, dpointL:dpointR);
            filter = dneighbor(se);
            extractmax(row, col) = max(filter);
        end
    end
    imwrite(extractmax, "scene_dil.jpg");
end
</syntaxhighlight>
| [[File:Lotus free dil.jpg|thumb|Denoising picture with dilation method]]
|-
| '''[[erosion (morphology)|Erosion]]'''
| align="center" | <math>
\begin{bmatrix}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{bmatrix}
</math>
| Use Matlab to erosion
<syntaxhighlight lang="matlab">
imwrite(image, 'scene_ero.jpg');
extractmin = zeros(size(image), class(image));
for i = 1: time
    ero_image = imread('scene_ero.jpg');
    for col = (halfW + 1): (c - halfW)
        for row = (halfH +1): (r -halfH)
            pointDown = row-halfH;
            pointUp = row+halfH;
            pointLeft = col-halfW;
            pointRight = col+halfW;
            neighbor = ero_image(pointDown:pointUp,pointLeft:pointRight);
            filter = neighbor(se);
            extractmin(row, col) = min(filter);
        end
    end
    imwrite(extractmin, "scene_ero.jpg");
end
</syntaxhighlight>
| [[File:Lotus free erosion.jpg|thumb|]]
|-
|-
| '''[[opening (morphology)|Opening]]'''
| align="center" | <math>
\begin{bmatrix}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{bmatrix}
</math>
| Use Matlab to Opening
<syntaxhighlight lang="matlab">
imwrite(extractmin, "scene_opening.jpg")
extractopen = zeros(size(image), class(image));
for i = 1 : time
    dil_image = imread('scene_opening.jpg');
    for col = (halfW + 1): (c - halfW)
        for row = (halfH + 1) : (r - halfH)
            dpointD = row - halfH;
            dpointU = row + halfH;
            dpointL = col - halfW;
            dpointR = col + halfW;
            dneighbor = dil_image(dpointD:dpointU, dpointL:dpointR);
            filter = dneighbor(se);
            extractopen(row, col) = max(filter);
        end
    end
    imwrite(extractopen, "scene_opening.jpg");
end
</syntaxhighlight>
| [[File:Lotus free opening.jpg|thumb|]]
|-
| '''[[closing (morphology)|Closing]]'''
| align="center" | <math>
\begin{bmatrix}
1 & 1 & 1 \\
1 & 1 & 1 \\
1 & 1 & 1
\end{bmatrix}
</math>
| Use Matlab to Closing
<syntaxhighlight lang="matlab">
imwrite(extractmax, "scene_closing.jpg")
extractclose = zeros(size(image), class(image));
for i = 1 : time
    ero_image = imread('scene_closing.jpg');
    for col = (halfW + 1): (c - halfW)
        for row = (halfH + 1) : (r - halfH)
            dpointD = row - halfH;
            dpointU = row + halfH;
            dpointL = col - halfW;
            dpointR = col + halfW;
            dneighbor = ero_image(dpointD:dpointU, dpointL:dpointR);
            filter = dneighbor(se);
            extractclose(row, col) = min(filter);
        end
    end
    imwrite(extractclose, "scene_closing.jpg");
end
</syntaxhighlight>
| [[File:Lotus free closing.jpg|thumb|Denoising picture with closing method]]
|-
|}