Editing Hamming code

{{distinguish|text=[[Huffman coding|Huffman code]]}}
{{short description|Family of linear error-correcting codes}}
{{More footnotes needed|date=March 2013}}

{{Infobox code
 | name           = Binary Hamming codes
 | image          = [[File:Hamming(7,4).svg|160px]]
 | image_caption  = The Hamming(7,4) code (with {{Math|''r'' {{=}} 3}})
 | namesake       = [[Richard W. Hamming]]
 | type           = [[Linear block code]]
 | block_length   = {{Math|2<sup>''r''</sup> − 1}} where {{Math|''r'' ≥ 2}}
 | message_length = {{Math|2<sup>''r''</sup> − ''r'' − 1}}
 | rate           = {{Math|1 − {{Sfrac|''r''|(2<sup>''r''</sup> − 1)}}}}
 | distance       = {{Math|3}}
 | alphabet_size  = {{Math|2}}
 | notation       = {{Math|[2<sup>''r''</sup> − 1, 2<sup>''r''</sup> − ''r'' − 1, 3]<sub>2</sub>}}-code
 | properties     = [[perfect code]]
}}

In [[computer science]] and [[telecommunications]], '''Hamming codes''' are a family of [[linear code|linear error-correcting codes]]. Hamming codes can detect one-bit and two-bit errors, or correct one-bit errors without detection of uncorrected errors. By contrast, the simple [[parity code]] cannot correct errors, and can detect only an odd number of bits in error. Hamming codes are [[perfect code]]s, that is, they achieve the highest possible [[Block code#The rate R|rate]] for codes with their [[block length]] and [[Block code#The distance d|minimum distance]] of three.<ref>[https://www.cs.cmu.edu/~venkatg/teaching/codingtheory/notes/notes1.pdf See Lemma 12 of]</ref>
[[Richard W. Hamming]] invented Hamming codes in 1950 as a way of automatically correcting errors introduced by [[punched card]] readers. In his original paper, Hamming elaborated his general idea, but specifically focused on the [[Hamming(7,4)]] code which adds three parity bits to four bits of data.{{Sfnp|Hamming|1950|pp=153&ndash;154}}

In [[mathematical]] terms, Hamming codes are a class of binary linear code. For each integer {{Math|''r'' ≥ 2}} there is a code-word with [[block length]] {{Math|''n'' {{=}} 2<sup>''r''</sup> − 1}} and [[Block code#The message length k|message length]] {{Math|''k'' {{=}} 2<sup>''r''</sup> − ''r'' − 1}}. Hence the rate of Hamming codes is {{Math|''R'' {{=}} ''k'' / ''n'' {{=}} 1 − ''r'' / (2<sup>''r''</sup> − 1)}}, which is the highest possible for codes with minimum distance of three (i.e., the minimal number of bit changes needed to go from any code word to any other code word is three) and block length {{Math|2<sup>''r''</sup> − 1}}. The [[parity-check matrix]] of a Hamming code is constructed by listing all columns of length {{Math|''r''}} that are non-zero, which means that the [[dual code]] of the Hamming code is the [[Hadamard code|shortened Hadamard code]], also known as a Simplex code. The parity-check matrix has the property that any two columns are pairwise [[linearly independent]].

Due to the limited redundancy that Hamming codes add to the data, they can only detect and correct errors when the error rate is low. This is the case in computer memory (usually RAM), where bit errors are extremely rare and Hamming codes are widely used, and a RAM with this correction system is an ECC RAM ([[ECC memory]]). In this context, an extended Hamming code having one extra parity bit is often used. Extended Hamming codes achieve a Hamming distance of four, which allows the decoder to distinguish between when at most one one-bit error occurs and when any two-bit errors occur. In this sense, extended Hamming codes are single-error correcting and double-error detecting, abbreviated as '''SECDED'''.

== History ==
[[Richard Hamming]], the inventor of Hamming codes, worked at [[Bell Labs]] in the late 1940s on the Bell [[Model V]] computer, an [[electromechanical]] relay-based machine with cycle times in seconds. Input was fed in on [[punched paper tape]], seven-eighths of an inch wide, which had up to six holes per row. During weekdays, when errors in the relays were detected, the machine would stop and flash lights so that the operators could correct the problem. During after-hours periods and on weekends, when there were no operators, the machine simply moved on to the next job.

Hamming worked on weekends, and grew increasingly frustrated with having to restart his programs from scratch due to detected errors. In a taped interview, Hamming said, "And so I said, 'Damn it, if the machine can detect an error, why can't it locate the position of the error and correct it?'".<ref>{{citation|last=Thompson|first=Thomas M.|title=From Error-Correcting Codes through Sphere Packings to Simple Groups|url=https://books.google.com/books?id=ggqxuG31B3cC&q=%22From%20Error-Correcting%20Codes%20through%20Sphere%20Packings%20to%20Simple%20Groups%22&pg=PA16|pages=16–17|year=1983|series=The Carus Mathematical Monographs (#21)|publisher=Mathematical Association of America|isbn=0-88385-023-0}}</ref> Over the next few years, he worked on the problem of error-correction, developing an increasingly powerful array of algorithms. In 1950, he published what is now known as Hamming code, which remains in use today in applications such as [[ECC memory]].

=== Codes predating Hamming ===
A number of simple error-detecting codes were used before Hamming codes, but none were as effective as Hamming codes in the same overhead of space.

==== Parity ====
{{main|Parity bit}}

Parity adds a single [[bit]] that indicates whether the number of ones (bit-positions with values of one) in the preceding data was [[even number|even]] or [[odd number|odd]]. If an odd number of bits is changed in transmission, the message will change parity and the error can be detected at this point; however, the bit that changed may have been the parity bit itself. The most common convention is that a parity value of one indicates that there is an odd number of ones in the data, and a parity value of zero indicates that there is an even number of ones. If the number of bits changed is even, the check bit will be valid and the error will not be detected.

Moreover, parity does not indicate which bit contained the error, even when it can detect it. The data must be discarded entirely and re-transmitted from scratch. On a noisy transmission medium, a successful transmission could take a long time or may never occur. However, while the quality of parity checking is poor, since it uses only a single bit, this method results in the least overhead.

====Two-out-of-five code====
{{main|Two-out-of-five code}}

A two-out-of-five code is an encoding scheme which uses five bits consisting of exactly three 0s and two 1s. This provides <math>\binom{5}{3}=10</math> possible combinations, enough to represent the digits 0–9. This scheme can detect all single bit-errors, all odd numbered bit-errors and some even numbered bit-errors (for example the flipping of both 1-bits). However it still cannot correct any of these errors.

====Repetition====
{{main|Triple modular redundancy}}

Another code in use at the time repeated every data bit multiple times in order to ensure that it was sent correctly. For instance, if the data bit to be sent is a 1, an {{nowrap|1=''n'' = 3}} ''[[repetition code]]'' will send 111. If the three bits received are not identical, an error occurred during transmission. If the channel is clean enough, most of the time only one bit will change in each triple. Therefore, 001, 010, and 100 each correspond to a 0 bit, while 110, 101, and 011 correspond to a 1 bit, with the greater quantity of digits that are the same ('0' or a '1') indicating what the data bit should be. A code with this ability to reconstruct the original message in the presence of errors is known as an ''error-correcting'' code. This triple repetition code is a Hamming code with {{nowrap|1=''m'' = 2,}} since there are two parity bits, and {{nowrap|1=2<sup>2</sup> − 2 − 1 = 1}} data bit.

Such codes cannot correctly repair all errors, however. In our example, if the channel flips two bits and the receiver gets 001, the system will detect the error, but conclude that the original bit is 0, which is incorrect. If we increase the size of the bit string to four, we can detect all two-bit errors but cannot correct them (the quantity of parity bits is even); at five bits, we can both detect and correct all two-bit errors, but not all three-bit errors.

Moreover, increasing the size of the parity bit string is inefficient, reducing throughput by three times in our original case, and the efficiency drops drastically as we increase the number of times each bit is duplicated in order to detect and correct more errors.

==Description==
If more error-correcting bits are included with a message, and if those bits can be arranged such that different incorrect bits produce different error results, then bad bits could be identified. In a seven-bit message, there are seven possible single bit errors, so three error control bits could potentially specify not only that an error occurred but also which bit caused the error.

Hamming studied the existing coding schemes, including two-of-five, and generalized their concepts. To start with, he developed a [[wikt:nomenclature|nomenclature]] to describe the system, including the number of data bits and error-correction bits in a block. For instance, parity includes a single bit for any data word, so assuming [[ASCII]] words with seven bits, Hamming described this as an ''(8,7)'' code, with eight bits in total, of which seven are data. The repetition example would be ''(3,1)'', following the same logic. The [[code rate]] is the second number divided by the first, for our repetition example, 1/3.

Hamming also noticed the problems with flipping two or more bits, and described this as the "distance" (it is now called the ''[[Hamming distance]]'', after him). Parity has a distance of 2, so one bit flip can be detected but not corrected, and any two bit flips will be invisible. The (3,1) repetition has a distance of 3, as three bits need to be flipped in the same triple to obtain another code word with no visible errors. It can correct one-bit errors or it can detect - but not correct - two-bit errors. A (4,1) repetition (each bit is repeated four times) has a distance of 4, so flipping three bits can be detected, but not corrected. When three bits flip in the same group there can be situations where attempting to correct will produce the wrong code word. In general, a code with distance ''k'' can detect but not correct {{nowrap|''k'' − 1}} errors.

Hamming was interested in two problems at once: increasing the distance as much as possible, while at the same time increasing the code rate as much as possible. During the 1940s he developed several encoding schemes that were dramatic improvements on existing codes. The key to all of his systems was to have the parity bits overlap, such that they managed to check each other as well as the data.

===General algorithm===
The following general algorithm generates a single-error correcting (SEC) code for any number of bits. The main idea is to choose the error-correcting bits such that the index-XOR (the [[XOR]] of all the bit positions containing a 1) is 0. We use positions 1, 10, 100, etc. (in binary) as the error-correcting bits, which guarantees it is possible to set the error-correcting bits so that the index-XOR of the whole message is 0. If the receiver receives a string with index-XOR 0, they can conclude there were no corruptions, and otherwise, the index-XOR indicates the index of the corrupted bit.

An algorithm can be deduced from the following description:

# Number the bits starting from 1: bit 1, 2, 3, 4, 5, 6, 7, etc.
# Write the bit numbers in binary:  1, 10, 11, 100, 101, 110, 111, etc.
# All bit positions that are powers of two (have a single 1 bit in the binary form of their position) are parity bits: 1, 2, 4, 8, etc. (1, 10, 100, 1000)
# All other bit positions, with two or more 1 bits in the binary form of their position, are data bits.
# Each data bit is included in a unique set of 2 or more parity bits, as determined by the binary form of its bit position.
## Parity bit 1 covers all bit positions which have the '''least''' significant bit set: bit 1 (the parity bit itself), 3, 5, 7, 9, etc.
## Parity bit 2 covers all bit positions which have the '''second''' least significant bit set: bits 2–3, 6–7, 10–11, etc.
## Parity bit 4 covers all bit positions which have the '''third''' least significant bit set: bits 4–7, 12–15, 20–23, etc.
## Parity bit 8 covers all bit positions which have the '''fourth''' least significant bit set: bits 8–15, 24–31, 40–47, etc.
## In general each parity bit covers all bits where the bitwise AND of the parity position and the bit position is non-zero.

If a byte of data to be encoded is 10011010, then the data word (using _ to represent the parity bits) would be __1_001_1010, and the code word is 011100101010.

The choice of the parity, even or odd, is irrelevant but the same choice must be used for both encoding and decoding.

This general rule can be shown visually:

:{| class="wikitable" style="text-align:center;"
|-
!colspan="2"| <span style="color:#888">Bit position</span>
! <span style="color:#888">1</span> !! <span style="color:#888">2</span> !! <span style="color:#888">3</span> !! <span style="color:#888">4</span> !! <span style="color:#888">5</span> !! <span style="color:#888">6</span> !! <span style="color:#888">7</span> !! <span style="color:#888">8</span> !! <span style="color:#888">9</span> !! <span style="color:#888">10</span> !! <span style="color:#888">11</span> !! <span style="color:#888">12</span> !! <span style="color:#888">13</span> !! <span style="color:#888">14</span> !! <span style="color:#888">15</span> !! <span style="color:#888">16</span> !! <span style="color:#888">17</span> !! <span style="color:#888">18</span> !! <span style="color:#888">19</span> !! <span style="color:#888">20</span>
|rowspan="7"| ...
|-
!colspan="2"| Encoded data bits
!style="background-color: #90FF90;"| p1
!style="background-color: #90FF90;"| p2 !! d1
!style="background-color: #90FF90;"| p4 !! d2 !! d3 !! d4
!style="background-color: #90FF90;"| p8 !! d5 !! d6 !! d7 !! d8 !! d9 !! d10 !! d11
!style="background-color: #90FF90;"| p16 !! d12 !! d13 !! d14 !! d15
|-
!rowspan="5"|Parity<br />bit<br />coverage
!style="background-color: #90FF90;"| p1
| {{ya}} ||   || {{ya}} ||   || {{ya}} ||   || {{ya}} ||   || {{ya}} ||   || {{ya}} ||   || {{ya}} ||   || {{ya}} ||   || {{ya}} ||   || {{ya}} || 
|-
!style="background-color: #90FF90;"| p2
|   || {{ya}} || {{ya}} ||   ||   || {{ya}} || {{ya}} ||   ||   || {{ya}} || {{ya}} ||   ||   || {{ya}} || {{ya}} ||   ||   || {{ya}} || {{ya}} || 
|-
!style="background-color: #90FF90;"| p4
|   ||   ||   || {{ya}} || {{ya}} || {{ya}} || {{ya}} ||   ||   ||   ||   || {{ya}} || {{ya}} || {{ya}} || {{ya}} ||   ||   ||   ||   || {{ya}}
|-
!style="background-color: #90FF90;"| p8
|   ||   ||   ||   ||   ||   ||   || {{ya}} || {{ya}} || {{ya}} || {{ya}} || {{ya}} || {{ya}} || {{ya}} || {{ya}} ||   ||   ||   ||   || 
|-
!style="background-color: #90FF90;"| p16
|   ||   ||   ||   ||   ||   ||   ||   ||   ||   ||   ||   ||   ||   ||   || {{ya}} || {{ya}} || {{ya}} || {{ya}} || {{ya}}
|}

Shown are only 20 encoded bits (5 parity, 15 data) but the pattern continues indefinitely.  The key thing about Hamming codes that can be seen from visual inspection is that any given bit is included in a unique set of parity bits.  To check for errors, check all of the parity bits.  The pattern of errors, called the [[error syndrome]], identifies the bit in error.  If all parity bits are correct, there is no error.  Otherwise, the sum of the positions of the erroneous parity bits identifies the erroneous bit.  For example, if the parity bits in positions 1, 2 and 8 indicate an error, then bit 1+2+8=11 is in error.  If only one parity bit indicates an error, the parity bit itself is in error.

With {{mvar|m}} parity bits, bits from 1 up to <math>2^m-1</math> can be covered. After discounting the parity bits, <math>2^m-m-1</math> bits remain for use as data. As {{mvar|m}} varies, we get all the possible Hamming codes:

{| class="wikitable" style="text-align:right"
|-
! Parity bits !! [[Block code#The block length n|Total bits]] !! [[Block code#The message length k|Data bits]] !! Name !! [[Block code#The rate R|Rate]]
|-
| 2 || 3 || 1|| Hamming(3,1)<br/>(Triple [[repetition code]]) || 1/3 ≈ 0.333
|-
| 3 || 7 || 4 || [[Hamming(7,4)]] || 4/7 ≈ 0.571
|-
| 4 || 15 || 11 || Hamming(15,11) || 11/15 ≈ 0.733
|-
| 5 || 31 || 26 ||Hamming(31,26) || 26/31 ≈ 0.839
|-
| 6 || 63 || 57 ||Hamming(63,57) || 57/63 ≈ 0.905
|-
| 7 || 127 || 120 ||Hamming(127,120) || 120/127 ≈ 0.945
|-
| 8 || 255 || 247 ||Hamming(255,247) || 247/255 ≈ 0.969
|-
| 9 || 511 || 502 ||Hamming(511,502) ||502/511 ≈ 0.982
|-
| colspan="5" | ...
|-
|{{mvar|m}}
|<math>n=2^m-1</math>
|<math>k=2^m-m-1</math>
|Hamming<math>(2^m-1,2^m-m-1)</math>
|<math>(2^m - m - 1)/(2^m-1)</math>
|}

== {{Anchor|SECDED}}Hamming codes with additional parity (SECDED) ==
Hamming codes have a minimum distance of 3, which means that the decoder can detect and correct a single error, but it cannot distinguish a double bit error of some codeword from a single bit error of a different codeword.  Thus, some double-bit errors will be incorrectly decoded as if they were single bit errors and therefore go undetected, unless no correction is attempted.

To remedy this shortcoming, Hamming codes can be extended by an extra parity bit. This way, it is possible to increase the minimum distance of the Hamming code to 4, which allows the decoder to distinguish between single bit errors and two-bit errors. Thus the decoder can detect and correct a single error and at the same time detect (but not correct) a double error.

If the decoder does not attempt to correct errors, it can reliably detect triple bit errors. If the decoder does correct errors, some triple errors will be mistaken for single errors and "corrected" to the wrong value. Error correction is therefore a trade-off between certainty (the ability to reliably detect triple bit errors) and resiliency (the ability to keep functioning in the face of single bit errors).

This extended Hamming code was popular in computer memory systems, starting with [[IBM 7030 Stretch]] in 1961,{{sfn|Kythe|Kythe|2017|p=115}} where it is known as ''SECDED'' (or SEC-DED, abbreviated from ''single error correction, double error detection'').{{sfn|Kythe|Kythe|2017|p=95}} Server computers in 21st century, while typically keeping the SECDED level of protection, no longer use Hamming's method, relying instead on the designs with longer codewords (128 to 256 bits of data) and modified balanced parity-check trees.{{sfn|Kythe|Kythe|2017|p=115}} The (72,64) Hamming code is still popular in some hardware designs, including [[Xilinx]] [[FPGA]] families.{{sfn|Kythe|Kythe|2017|p=115}}

==[7,4] Hamming code==
{{main|Hamming(7,4)}}
[[File:Hamming(7,4).svg|thumb|300px|Graphical depiction of the four data bits and three parity bits and which parity bits apply to which data bits]]

In 1950, Hamming introduced the [7,4] Hamming code. It encodes four data bits into seven bits by adding three parity bits. As explained earlier, it can either detect and correct single-bit errors or it can detect (but not correct) both single and double-bit errors.

With the addition of an overall parity bit, it becomes the [8,4] extended Hamming code and can both detect and correct single-bit errors and detect (but not correct) double-bit errors.

===Construction of G and H===
The matrix 
<math>\mathbf{G} := \begin{pmatrix} \begin{array}{c|c}
I_k & -A^\text{T} \\
\end{array} \end{pmatrix}</math> is called a (canonical) generator matrix of a linear (''n'',''k'') code,

and <math>\mathbf{H} := \begin{pmatrix} \begin{array}{c|c}
A & I_{n-k} \\
\end{array} \end{pmatrix}</math> is called a [[parity-check matrix]].

This is the construction of '''G''' and '''H''' in standard (or systematic) form. Regardless of form, '''G''' and '''H''' for linear block codes must satisfy

<math>\mathbf{H}\,\mathbf{G}^\text{T} = \mathbf{0}</math>, an all-zeros matrix.<ref name=Moon>Moon T. Error correction coding: Mathematical Methods and
Algorithms. John Wiley and Sons, 2005.(Cap. 3) {{ISBN|978-0-471-64800-0}}</ref>

Since [7,&nbsp;4,&nbsp;3] =&nbsp;[''n'',&nbsp;''k'',&nbsp;''d''] =&nbsp;[2<sup>''m''</sup>&nbsp;−&nbsp;1, 2<sup>''m''</sup>&nbsp;−&nbsp;1&nbsp;−&nbsp;''m'',&nbsp;3]. The [[parity-check matrix]] '''H''' of a Hamming code is constructed by listing all columns of length ''m'' that are pair-wise independent.

Thus '''H''' is a matrix whose left side is all of the nonzero ''n''-tuples where order of the ''n''-tuples in the columns of matrix does not matter. The right hand side is just the (''n''&nbsp;−&nbsp;''k'')-[[identity matrix]].

So '''G''' can be obtained from '''H''' by taking the transpose of the left hand side of '''H''' with the identity ''k''-[[identity matrix]] on the left hand side of&nbsp;'''G'''.

The code [[generator matrix]] <math>\mathbf{G}</math> and the [[parity-check matrix]] <math>\mathbf{H}</math> are:

<math>\mathbf{G} := \begin{pmatrix}
1 & 0 & 0 & 0 & 1 & 1 & 0 \\
0 & 1 & 0 & 0 & 1 & 0 & 1 \\
0 & 0 & 1 & 0 & 0 & 1 & 1 \\
0 & 0 & 0 & 1 & 1 & 1 & 1
\end{pmatrix}_{4,7}</math>

and

<math>\mathbf{H} := \begin{pmatrix}
1 & 1 & 0 & 1 & 1 & 0 & 0 \\
1 & 0 & 1 & 1 & 0 & 1 & 0 \\
0 & 1 & 1 & 1 & 0 & 0 & 1
\end{pmatrix}_{3,7}.</math>

Finally, these matrices can be mutated into equivalent non-systematic codes by the following operations:<ref name=Moon/>

* Column permutations (swapping columns)
* Elementary row operations (replacing a row with a linear combination of rows)

===Encoding===
;Example

From the above matrix we have 2<sup>k</sup> = 2<sup>4</sup> = 16 codewords.
Let <math>\vec{a}</math> be a row vector of binary data bits, <math> \vec{a}=[a_1,a_2,a_3,a_4],\quad a_i\in \{0,1\} </math>. The codeword <math>\vec{x}</math> for any of the 16 possible data vectors <math> \vec{a} </math> is given by the standard matrix product <math>\vec{x}=\vec{a}G </math> where the summing operation is done modulo-2.

For example, let <math>\vec{a}=[1,0,1,1]</math>. Using the generator matrix <math>G</math> from above, we have (after applying modulo 2, to the sum),

<math>\vec{x}=\vec{a}G= \begin{pmatrix} 1 & 0 & 1 & 1 \end{pmatrix}\begin{pmatrix}
1 & 0 & 0 & 0 & 1 & 1 & 0 \\
0 & 1 & 0 & 0 & 1 & 0 & 1 \\
0 & 0 & 1 & 0 & 0 & 1 & 1 \\
0 & 0 & 0 & 1 & 1 & 1 & 1 \\
\end{pmatrix} = \begin{pmatrix} 1 & 0 & 1 & 1 & 2 & 3 & 2 \end{pmatrix} = \begin{pmatrix} 1 & 0 & 1 & 1 & 0 & 1 & 0 \end{pmatrix}</math>

===[8,4] Hamming code with an additional parity bit===
[[File:Hamming(8,4).svg|thumb|300px|The same [7,4] example from above with an extra parity bit. This diagram is not meant to correspond to the matrix H for this example.]]

The [7,4] Hamming code can easily be extended to an [8,4] code by adding an extra parity bit on top of the (7,4) encoded word (see [[Hamming(7,4)]]).
This can be summed up with the revised matrices:

:<math>\mathbf{G} := \begin{pmatrix}
1 & 1 & 1 & 0 & 0 & 0 & 0 & 1\\
1 & 0 & 0 & 1 & 1 & 0 & 0 & 1\\
0 & 1 & 0 & 1 & 0 & 1 & 0 & 1\\
1 & 1 & 0 & 1 & 0 & 0 & 1 & 0
\end{pmatrix}_{4,8}</math>

and

:<math>
\mathbf{H} :=
\begin{pmatrix}
1 & 0 & 1 & 0 & 1 & 0 & 1 & 0\\
0 & 1 & 1 & 0 & 0 & 1 & 1 & 0\\
0 & 0 & 0 & 1 & 1 & 1 & 1 & 0\\
1 & 1 & 1 & 1 & 1 & 1 & 1 & 1
\end{pmatrix}_{4,8}
.</math>

<!-- the equations here before had two (2) errors--these satisfy G*H^T=0 -->

<!-- I don't have the time to fix it myself right now, but in the format G is in after row-reduction, it changes the order -->
<!-- of the bits in the message being delivered... i.e., any of the columns of G with one hi bit will pass the data directly -->
<!-- Ex. [d1 d2 d3 d4] * G = [d1 d2 d3 d4 p1 p2 p3 p4], this is not the same as the format described immediately below it -->
<!-- where the message should be in the format [p1 p2 d1 p3 d2 d3 d4 p4] -->

Note that H is not in standard form. To obtain G, elementary row operations can be used to obtain an equivalent matrix to H in systematic form:

:<math>
\mathbf{H} =
\left( \begin{array}{cccc|cccc}
0 & 1 & 1 & 1  &  1 & 0 & 0 & 0 \\
1 & 0 & 1 & 1  &  0 & 1 & 0 & 0 \\
1 & 1 & 0 & 1  &  0 & 0 & 1 & 0 \\
1 & 1 & 1 & 0  &  0 & 0 & 0 & 1 \end{array} \right)_{4,8}
.</math>

For example, the first row in this matrix is the sum of the second and third rows of H in non-systematic form. Using the systematic construction for Hamming codes from above, the matrix A is apparent and the systematic form of G is written as

:<math>
\mathbf{G} =
\left( \begin{array}{cccc|cccc}
1 & 0 & 0 & 0  &  0 & 1 & 1 & 1 \\
0 & 1 & 0 & 0  &  1 & 0 & 1 & 1 \\
0 & 0 & 1 & 0  &  1 & 1 & 0 & 1 \\
0 & 0 & 0 & 1  &  1 & 1 & 1 & 0 \end{array} \right)_{4,8}
.</math>

The non-systematic form of G can be row reduced (using elementary row operations) to match this matrix.

The addition of the fourth row effectively computes the sum of all the codeword bits (data and parity) as the fourth parity bit.

For example, <span style="color:blue;">1011</span> is encoded (using the non-systematic form of G at the start of this section) into <span style="color:red;">01</span><span style="color:blue;">1</span><span style="color:red;">0</span><span style="color:blue;">011</span><span style="color:green;">0</span> where blue digits are data; red digits are parity bits from the [7,4] Hamming code; and the green digit is the parity bit added by the [8,4] code.
The green digit makes the parity of the [7,4] codewords even.

Finally, it can be shown that the minimum distance has increased from 3, in the [7,4] code, to 4 in the [8,4] code. Therefore, the code can be defined as [8,4] Hamming code. <!-- if we need more proof, or a citation, let me know -->

To decode the [8,4] Hamming code, first check the parity bit. If the parity bit indicates an error, single error correction (the [7,4] Hamming code) will indicate the error location, with "no error" indicating the parity bit. If the parity bit is correct, then single error correction will indicate the (bitwise) exclusive-or of two error locations. If the locations are equal ("no error") then a double bit error either has not occurred, or has cancelled itself out. Otherwise, a double bit error has occurred.

== See also ==

{{Div col|colwidth=20em}}
* [[Coding theory]]
* [[Golay code (disambiguation)|Golay code]]
* [[Hamming bound]]
* [[Hamming distance]]
* [[Low-density parity-check code]]
* [[Reed–Muller code]]
* [[Reed–Solomon error correction]]
* [[Turbo code]]
{{div col end}}

== Notes ==
{{Reflist}}

== References ==
{{refbegin}}
* {{cite journal|last1=Hamming|first1=Richard Wesley|title=Error detecting and error correcting codes|journal=[[Bell System Technical Journal]]|date=1950|volume=29|issue=2|pages=147–160|doi=10.1002/j.1538-7305.1950.tb00463.x|hdl=10945/46756 |s2cid=61141773 |url=https://calhoun.nps.edu/bitstream/10945/46756/1/Hamming_1982.pdf |archive-url=https://ghostarchive.org/archive/20221009/https://calhoun.nps.edu/bitstream/10945/46756/1/Hamming_1982.pdf |archive-date=2022-10-09 |url-status=live}}
* {{cite book
  |last=Moon
  |first=Todd K.
  |title=Error Correction Coding
  |publisher=[[John Wiley & Sons]]
  |year=2005
  |location=[[New Jersey]]
  |url=http://www.neng.usu.edu/ece/faculty/tmoon/eccbook/book.html
  |isbn= 978-0-471-64800-0
}}
* {{cite book
  |last=MacKay
  |first=David J.C.
  |author-link=David MacKay (scientist)
  |title=Information Theory, Inference and Learning Algorithms
  |publisher=[[Cambridge University Press]]
  |date=September 2003
  |location=[[Cambridge]]
  |url=http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
  |isbn= 0-521-64298-1}}
* {{cite conference
  |author=D.K. Bhattacharryya, S. Nandi
  |title=An efficient class of SEC-DED-AUED codes
  |pages=410–415
  |book-title=1997 International Symposium on Parallel Architectures, Algorithms and Networks (ISPAN '97)
  |doi=10.1109/ISPAN.1997.645128
}}
* {{cite web
  |title=Mathematical Challenge April 2013 Error-correcting codes
  |publisher=[[swissQuant Group Leadership Team]]
  |date=April 2013
  |url=https://www.swissquant.com/wp-content/uploads/2017/09/Mathematical-Challenge-April-2013.pdf |archive-url=https://web.archive.org/web/20170912191717/https://www.swissquant.com/wp-content/uploads/2017/09/Mathematical-Challenge-April-2013.pdf |archive-date=2017-09-12 |url-status=live
  }}
* {{cite book | first1 = Dave K. | last1 = Kythe | first2 = Prem K. | last2 = Kythe | date = 28 July 2017 | title = Algebraic and Stochastic Coding Theory | chapter = Extended Hamming Codes | publisher = CRC Press | pages = 95–116 | isbn = 978-1-351-83245-8 | chapter-url = https://books.google.com/books?id=zJwuDwAAQBAJ&pg=PA95}}
{{refend}}

== External links ==
* [https://www.youtube.com/watch?v=X8jsijhllIA Visual Explanation of Hamming Codes]
* [http://www.ee.unb.ca/cgi-bin/tervo/hamming.pl CGI script for calculating Hamming distances (from R. Tervo, UNB, Canada)]
* [http://www.toolmenow.com/34/Hamming(7,4)-Code-Calculator Tool for calculating Hamming code]

{{DEFAULTSORT:Hamming Code}}
[[Category:American inventions]]
[[Category:Coding theory]]
[[Category:Error detection and correction]]
[[Category:Computer arithmetic]]
[[Category:1951 in computing]]