Editing Tar (computing) (section)

==File format==
There are multiple tar file formats, including historical and current ones. Two tar formats are codified in POSIX: [[#UStar format|ustar]] and [[#POSIX.1-2001/pax|pax]]. Not codified but still in current use is the GNU tar format.

A tar archive consists of a series of file objects, hence the popular term ''tarball'', referencing how a [[Tarball (oil)|tarball]]  collects objects of all kinds that stick to its surface.  Each file object includes any file data, and is preceded by a 512-byte [[Header (computing)|header]] record.  The file data is written unaltered except that its length is rounded up to a multiple of 512 bytes. The original tar implementation did not care about the contents of the padding bytes, and left the buffer data unaltered, but most modern tar implementations fill the extra space with zeros.<ref name = "Hoo, e7z Org., 2015" >{{ Cite web | url = http://www.e7z.org/open-tar.htm | title = Open/Extract TAR File with Freeware on Windows/Mac/Linux | access-date = 2019-09-03 | first = James | last = Hoo | website = e7z Org. | archive-url = https://web.archive.org/web/20150206035752/http://www.e7z.org/open-tar.htm | archive-date = 2015-02-06 | df = dmy-all }}</ref> The end of an archive is marked by at least two consecutive zero-filled records.  (The origin of tar's record size appears to be the 512-byte disk sectors used in the Version 7 Unix file system.) The final block of an archive is padded out to full length with zeros.

===Header===
The file header record contains [[metadata]] about a file.  To ensure portability across different architectures with different [[Endianness|byte orderings]], the information in the header record is encoded in [[ASCII]].  Thus if all the files in an archive are ASCII text files, and have ASCII names, then the archive is essentially an ASCII text file (containing many [[Null character|NUL characters]]).

The fields defined by the original Unix tar format are listed in the table below.  The link indicator/file type table includes some modern extensions.  When a field is unused it is filled with NUL bytes.  The header uses 257 bytes, then is padded with NUL bytes to make it fill a 512 byte record.  There is no "magic number" in the header, for file identification.

Pre-POSIX.1-1988 (i.e. v7) tar header:
{| class="wikitable"
|-
! Field offset
! Field size
! Field
|-
| 0
| 100
| File path and name
|-
| 100
| 8
| File mode (octal)
|-
| 108
| 8
| Owner's numeric user ID (octal)
|-
| 116
| 8
| Group's numeric user ID (octal)
|-
| 124
| 12
| File size in bytes (octal)
|-
| 136
| 12
| Last modification time in numeric Unix time format (octal)
|-
| 148
| 8
| Checksum for header record
|-
| 156
| 1
| Link indicator (file type)
|-
| 157
| 100
| Name of linked file
|}

The pre-POSIX.1-1988 '''Link indicator''' field can have the following values:

{| class="wikitable"
|+ Link indicator field
|-
! Value
! Meaning
|-
| '0' or ([[ASCII]] [[Null character|NUL]])
| Normal file
|-
| '1'
| [[Hard link]]
|-
| '2'
| [[Symbolic link]]
|}

Some pre-POSIX.1-1988 tar implementations indicated a directory by having a trailing [[Slash (punctuation)|slash]] (/) in the name.

Numeric values are encoded in [[octal]] numbers using ASCII digits, with leading zeroes.  For historical reasons, a final NUL or [[Space (punctuation)|space]] character should also be used. Thus although there are 12 bytes reserved for storing the file size, only 11 octal digits can be stored.  This gives a maximum file size of 8 [[gigabyte]]s on archived files.  To overcome this limitation, in 2001 star introduced a base-256 coding that is indicated by setting the high-order bit of the leftmost byte of a numeric field.{{cn|date=October 2021}} GNU-tar and BSD-tar followed this idea. Additionally, versions of tar from before the first POSIX standard from 1988 pad the values with spaces instead of zeroes.

The [[checksum]] is calculated by taking the sum of the unsigned byte values of the header record with the eight checksum bytes taken to be ASCII spaces (decimal value 32). It is stored as a six digit octal number with leading zeroes followed by a NUL and then a space. Various implementations do not adhere to this format. In addition, some historic tar implementations treated bytes as signed. Implementations typically calculate the checksum both ways, and treat it as good if either the signed or unsigned sum matches the included checksum.

Unix filesystems support multiple links (names) for the same file.  If several such files appear in a tar archive, only the first one is archived as a normal file; the rest are archived as hard links, with the "name of linked file" field set to the first one's name.  On extraction, such hard links should be recreated in the file system.

===UStar format===
Most modern tar programs read and write archives in the UStar (''Unix Standard TAR''<ref name="tar5"/><ref>{{cite book |last=Kientzle |first=Tim |date=1995 |title=Internet File Formats |publisher=Coriolis Groups Books |isbn=978-1-883577-56-8 |url=https://archive.org/details/mac_Internet_File_Formats_1995 |page=[https://archive.org/details/mac_Internet_File_Formats_1995/page/n202 196] |access-date=2022-11-10}}</ref>) format, introduced by the POSIX IEEE P1003.1 standard from 1988.  It introduced additional header fields.  Older tar programs will ignore the extra information (possibly extracting partially named files), while newer programs will test for the presence of the "ustar" string to determine if the new format is in use.  

The UStar format allows for longer file names and stores additional information about each file.  The maximum filename size is 255, but it is split among a preceding path "filename prefix" and the filename itself, so can be much less.<ref name="gnu.org">{{Cite web | url=https://www.gnu.org/software/tar/manual/html_chapter/Formats.html#Compression|title = GNU tar 1.32: 8.1 Using Less Space through Compression | date = 2019-02-23 | website = GNU | access-date = 2019-08-11}}</ref>

{| class="wikitable"
|-
! Field offset
! Field size
! Field
|-
| 0
| 156
| ''(Several fields, same as in the old format)''
|-
| 156
| 1
| Type flag
|-
| 157
| 100
| ''(Same field as in the old format)''
|-
| 257
| 6
| UStar indicator, "ustar", then NUL
|-
| 263
| 2
| UStar version, "00"
|-
| 265
| 32
| Owner user name
|-
| 297
| 32
| Owner group name
|-
| 329
| 8
| Device major number
|-
| 337
| 8
| Device minor number
|-
| 345
| 155
| Filename prefix
|}

The ''type flag'' field can have the following values:

{| class="wikitable"
|+ Type flag field
|-
! Value
! Meaning
|-
| '0' or ([[ASCII]] [[Null character|NUL]])
| Normal file
|-
| '1'
| [[Hard link]]
|-
| '2'
| [[Symbolic link]]
|-
| '3'
| [[Device file|Character special]]
|-
| '4'
| [[Device file|Block special]]
|-
| '5'
| [[Directory (computing)|Directory]]
|-
| '6'
| [[Named pipe|FIFO]]
|-
| '7'
| Contiguous file
|-
| 'g'
| Global extended header with meta data (POSIX.1-2001)
|-
| 'x'
| Extended header with metadata for the next file in the archive (POSIX.1-2001)
|-
| 'A'–'Z'
| Vendor specific extensions (POSIX.1-1988)
|-
| All other values
| Reserved for future standardization
|}

POSIX.1-1988 vendor specific extensions using link flag values 'A'–'Z' partially have a different meaning with different vendors and thus are seen as outdated and replaced by the POSIX.1-2001 extensions that also include a vendor tag.

Type '7' (Contiguous file) is formally marked as reserved in the POSIX standard, but was meant to indicate files which ought to be contiguously allocated on disk. Few operating systems support creating such files explicitly, and hence most TAR programs do not support them, and will treat type 7 files as if they were type 0 (regular). An exception is older versions of GNU tar, when running on the [[MASSCOMP]] RTU (Real Time Unix) operating system, which supported an O_CTG flag to the open() function to request a contiguous file; however, that support was removed from GNU tar version 1.24 onwards.

===POSIX.1-2001/pax===
In 1997, [[Sun Microsystems|Sun]] proposed a method for adding extensions to the tar format. This method was later accepted for the POSIX.1-2001 standard. This format is known as ''extended tar'' format or [[pax (command)|pax]] format. The new tar format allows users to add any type of vendor-tagged vendor-specific enhancements. The following tags are defined by the POSIX standard:
* '''atime''', '''mtime''': all timestamps of a file in arbitrary resolution (most implementations use nanosecond granularity)
* '''path''': path names of unlimited length and character set coding
* '''linkpath''': symlink target names of unlimited length and character set coding
* '''uname''', '''gname''': user and group names of unlimited length and character set coding
* '''size''': files with unlimited size (the historic tar format is 8 GB)
* '''uid''', '''gid''': userid and groupid without size limitation (the historic tar format is limited to a max. id of 2097151)
* a character set definition for path names and user/group names ([[UTF-8]])

In 2001, the Star program became the first tar to support the new format.{{cn|date=October 2021}} In 2004, GNU tar supported the new format,<ref>[https://git.savannah.gnu.org/cgit/tar.git/tree/NEWS NEWS], git.savannah.gnu.org - search for "Added support for POSIX.1-2001 and ustar archive formats."</ref> though it does not write it as its default output from the tar program yet.<ref name="gnu.org2">{{Cite web | url=https://www.gnu.org/software/tar/manual/html_section/Formats.html |title = GNU tar 1.34: 8. Controlling the Archive Format | website = GNU | access-date = 2022-07-11}}</ref>

The pax format is designed so that all implementations able to read the UStar format will be able to read the pax format as well. The only exceptions are files that make use of extended features, such as longer file names. For compatibility, these are encoded in the tar files as special {{code|x}} or {{code|g}} type files, typically under a {{code|PaxHeaders.XXXX}} directory.<ref>{{man|1|pax|SUS}}</ref>{{rp|exthdr.name}} A pax-supporting implementation would make use of the information, while non-supporting ones like [[7-Zip]] would process them as additional files.<ref>{{cite web |title=#2116 Tars with pax headers not parsed |url=https://sourceforge.net/p/sevenzip/bugs/2116/ |website=7-Zip / Bugs {{!}} SourceForge}}</ref>