Editing Netfilter

{{short description|Packet alteration framework for Linux and the umbrella project for software of the same}}

{{Infobox software
| name = Linux <!-- This is the name used to transclude the {{Latest stable software release/Linux}} and {{Latest preview software release/Linux}} version templates. -->
| title = Netfilter
| logo = 
| screenshot = 
| caption = 
| developer = 
| released = {{Start date and age|1999|08|26|df=yes}} (Linux 2.3.15)
| author = 
| latest release version = {{wikidata|property|edit|reference|Q14579|P548=Q2804309|P348}}
| latest release date = {{start date and age|{{wikidata|qualifier|Q14579|P548=Q2804309|P348|P577}}}}
| latest preview version = <!-- {{wikidata|property|edit|reference|Q14579|P548=Q51930650|P348}} -->
| latest preview date = {{start date and age|{{wikidata|qualifier|Q14579|P548=Q51930650|P348|P577}}}}
| operating system = [[Linux]]
| programming language = [[C (programming language)|C]]
| genre = {{ubl|[[Linux kernel module]]|[[Packet filter]]/firewall}}
| license = [[GNU&nbsp;GPL]]
| website = {{URL|https://netfilter.org}}
}}

'''Netfilter''' is a [[software framework|framework]] provided by the [[Linux kernel]] that allows various [[computer network|networking]]-related operations to be implemented in the form of customized handlers. Netfilter offers various functions and operations for [[packet filter]]ing, [[network address translation]], and [[port translation]], which provide the functionality required for directing packets through a network and [[firewall (computing)|prohibiting]] packets from reaching sensitive locations within a network.

Netfilter represents a set of [[hooking|hooks]] inside the Linux kernel, allowing specific [[kernel module]]s to register [[callback (computer programming)|callback]] functions with the kernel's networking stack. Those functions, usually applied to the traffic in the form of filtering and modification rules, are called for every packet that traverses the respective hook within the networking stack.<ref>{{cite web
 | url        = https://www.netfilter.org/
 | title      = netfilter/iptables project homepage - The netfilter.org project
 | publisher  = netfilter.org
 | access-date = 2014-07-04}}</ref>

==History==
[[Image:Netfilter-components.svg|300px|thumb|Relation of (some of) the different Netfilter components]]

[[Rusty Russell]] started the ''netfilter/iptables project'' in 1998; he had also authored the project's predecessor, [[ipchains]]. As the project grew, he founded the ''Netfilter Core Team'' (or simply ''coreteam'') in 1999. The software they produced (called ''netfilter'' hereafter) uses the [[GNU General Public License]] (GPL) license, and on 26 August 1999 it was merged into version 2.3.15 of the [[Linux kernel mainline]] and thus was in the 2.4.0 stable version.<ref name=":0">[[Harald Welte]], [https://netdevconf.info/2.2/slides/welte-netfilterhistory-keynote.pdf netfilter archeology: 18 years from 2.3 to 4.x], 5 December 2017</ref>

In August 2003 [[Harald Welte]] became chairman of the coreteam. In April 2004, following a crack-down by the project on those distributing the project's software [[embedded system|embedded]] in [[router (computing)|router]]s without complying with the GPL, a [[Germany|German]] court granted Welte an historic [[injunction]] against [[Sitecom]] Germany, which refused to follow the GPL's terms (see [[GNU General Public License#Legal status|GPL-related disputes]]). In September 2007 Patrick McHardy, who led development for past years, was elected as new chairman of the coreteam.

Prior to iptables, the predominant software packages for creating Linux firewalls were [[ipchains]] in Linux kernel 2.2.x and [[ipfwadm]] in Linux kernel 2.0.x,<ref name=":0" /> which in turn was based on [[Berkeley Software Distribution|BSD]]'s [[ipfirewall|ipfw]]. Both ipchains and ipfwadm alter the networking code so they can manipulate packets, as Linux kernel lacked a general packets control framework until the introduction of Netfilter.

Whereas ipchains and ipfwadm combine packet filtering and NAT (particularly three specific kinds of [[Network address translation|NAT]], called ''masquerading'', ''port forwarding'', and ''redirection''), Netfilter separates packet operations into multiple parts, described below. Each connects to the Netfilter hooks at different points to access packets. The connection tracking and NAT subsystems are more general and more powerful than the rudimentary versions within ipchains and ipfwadm.

In 2017 [[IPv4]] and [[IPv6]] flow offload infrastructure was added, allowing a speedup of software flow table forwarding and hardware offload support.<ref name="flow-offload-infrastructure-1">{{cite web
 | url     = https://lwn.net/Articles/738214/
 | title   = Flow offload infrastructure
 | website = [[LWN.net]]}}</ref><ref name="flow-offload-infrastructure-2">{{cite web
 | url     = https://lwn.net/Articles/742164/
 | title   = Flow offload infrastructure
 | website = [[LWN.net]]}}</ref>

==Userspace utility programs==
[[File:Netfilter-packet-flow.svg|thumb|right|600px|Flow of network packets through Netfilter with legacy iptables packet filtering]]

* {{man|8|iptables|man.cx||inline}}
* {{man|8|ip6tables|man.cx||inline}}
* {{man|8|ebtables|man.cx||inline}}
* {{man|8|arptables|man.cx||inline}}
* {{man|8|ipset|man.cx||inline}}
* {{man|8|nftables|man.cx||inline}}

===iptables===
{{Main|iptables}}

The kernel modules named <code>ip_tables</code>, <code>ip6_tables</code>, <code>arp_tables</code> (the underscore is part of the name), and <code>ebtables</code> comprise the legacy packet filtering portion of the Netfilter hook system. They provide a table-based system for defining firewall rules that can filter or transform packets. The tables can be administered through the user-space tools <code>iptables</code>, <code>ip6tables</code>, <code>arptables</code>, and <code>ebtables</code>. Notice that although both the kernel modules and userspace utilities have similar names, each of them is a different entity with different functionality.

Each table is actually its own hook, and each table was introduced to serve a specific purpose. As far as Netfilter is concerned, it runs a particular table in a specific order with respect to other tables. Any table can call itself and it also can execute its own rules, which enables possibilities for additional processing and iteration.

Rules are organized into chains, or in other words, "chains of rules". These chains are named with predefined titles, including <code>INPUT</code>, <code>OUTPUT</code> and <code>FORWARD</code>. These chain titles help describe the origin in the Netfilter stack. Packet reception, for example, falls into <code>PREROUTING</code>, while the <code>INPUT</code> represents locally delivered data, and forwarded traffic falls into the <code>FORWARD</code> chain. Locally generated output passes through the <code>OUTPUT</code> chain, and packets to be sent out are in <code>POSTROUTING</code> chain.

Netfilter modules not organized into tables (see below) are capable of checking for the origin to select their mode of operation.

; <code>iptable_raw</code> module
: When loaded, registers a hook that will be called before any other Netfilter hook. It provides a table called ''raw'' that can be used to filter packets before they reach more memory-demanding operations such as Connection Tracking.
;<code>iptable_mangle</code> module
: Registers a hook and ''mangle'' table to run after Connection Tracking (see below) (but still before any other table), so that modifications can be made to the packet. This enables additional modifications by rules that follow, such as NAT or further filtering.
; <code>iptable_nat</code> module
: Registers two hooks: Destination Network Address Translation-based transformations ("DNAT") are applied before the filter hook, Source Network Address Translation-based transformations ("SNAT") are applied afterwards. The ''network address translation'' table (or "nat") that is made available to iptables is merely a "configuration database" for [[Network address translation|NAT]] mappings only, and not intended for filtering of any kind.
; <code>iptable_filter</code> module
: Registers the ''filter'' table, used for general-purpose filtering (firewalling).
; <code>security_filter</code> module
: Used for Mandatory Access Control (MAC) networking rules, such as those enabled by the <code>SECMARK</code> and <code>CONNSECMARK</code> targets. (These so-called "targets" refer to Security-Enhanced Linux markers.)  Mandatory Access Control is implemented by Linux Security Modules such as SELinux. The security table is called following the call of the filter table, allowing any Discretionary Access Control (DAC) rules in the filter table to take effect before any MAC rules. This table provides the following built-in chains: <code>INPUT</code> (for packets coming into the computer itself), <code>OUTPUT</code> (for altering locally-generated packets before routing), and <code>FORWARD</code> (for altering packets being routed through the computer).

===nftables===
{{Main|nftables}}

nftables is the new packet-filtering portion of Netfilter. <code>nft</code> is the new userspace utility that replaces <code>iptables</code>, <code>ip6tables</code>, <code>arptables</code> and <code>ebtables</code>.

nftables kernel engine adds a simple [[virtual machine]] into the Linux kernel, which is able to execute bytecode to inspect a network packet and make decisions on how that packet should be handled. The operations implemented by this virtual machine are intentionally made basic: it can get data from the packet itself, have a look at the associated metadata (inbound interface, for example), and manage connection tracking data. Arithmetic, bitwise and comparison operators can be used for making decisions based on that data. The virtual machine is also capable of manipulating sets of data (typically IP addresses), allowing multiple comparison operations to be replaced with a single set lookup.<ref name="lwn-nftables">{{cite web
 | url        = https://lwn.net/Articles/564095/
 | title      = The return of nftables
 | date       = 2013-08-20
 | access-date = 2013-10-22
 | author     = Jonathan Corbet
 | publisher  = [[LWN.net]]}}</ref>

This is in contrast to the legacy Xtables (iptables, etc.) code, which has protocol awareness so deeply built into the code that it has had to be replicated four times{{mdashb}}for IPv4, IPv6, ARP, and Ethernet bridging{{mdashb}}as the firewall engines are too protocol-specific to be used in a generic manner.<ref name="lwn-nftables" /> The main advantages over <code>iptables</code> are simplification of the Linux kernel [[Application binary interface|ABI]], reduction of [[duplicate code|code duplication]], improved [[error message|error reporting]], and more efficient execution, storage, and incremental, [[Atomicity_(database_systems)|atomic]] changes of filtering rules.

==Packet defragmentation==
{{See also|IP fragmentation}}

The <code>nf_defrag_ipv4</code> module will defragment IPv4 packets before they reach Netfilter's connection tracking (<code>nf_conntrack_ipv4</code> module). This is necessary for the in-kernel connection tracking and NAT helper modules (which are a form of "mini-[[Application-level gateway|ALGs]]") that only work reliably on entire packets, not necessarily on fragments.

The IPv6 defragmenter is not a module in its own right, but is integrated into the <code>nf_conntrack_ipv6</code> module.

==Connection tracking==
{{See also|Stateful packet inspection}}

One of the important features built on top of the Netfilter framework is connection tracking.<ref>{{cite web
 | title = Netfilter's Connection Tracking System
 | first = Pablo
 | last  = Neira Ayuso
 | date  = 14 June 2006
 | url   = https://people.netfilter.org/pablo/docs/login.pdf}}</ref> Connection tracking allows the kernel to keep track of all logical network connections or [[Session (computer science)|sessions]], and thereby relate all of the packets which may make up that connection. NAT relies on this information to translate all related packets in the same way, and <code>iptables</code> can use this information to act as a stateful firewall.

The connection state however is completely independent of any upper-level state, such as TCP's or SCTP's state. Part of the reason for this is that when merely forwarding packets, i.e. no local delivery, the TCP engine may not necessarily be invoked at all. Even [[connectionless-mode transmission]]s such as [[User Datagram Protocol|UDP]], [[IPsec]] (AH/ESP), [[Generic Routing Encapsulation|GRE]] and other [[tunneling protocol]]s have, at least, a pseudo connection state. The heuristic for such protocols is often based upon a preset timeout value for inactivity, after whose expiration a Netfilter connection is dropped.

Each Netfilter connection is uniquely identified by a (layer-3 protocol, source address, destination address, layer-4 protocol, layer-4 key) tuple. The layer-4 key depends on the transport protocol; for TCP/UDP it is the port numbers, for tunnels it can be their tunnel ID, but otherwise is just zero, as if it were not part of the tuple. To be able to inspect the TCP port in all cases, packets will be mandatorily defragmented.

Netfilter connections can be manipulated with the user-space tool <code>conntrack</code>.

<code>iptables</code> can make use of checking the connection's information such as states, statuses and more to make packet filtering rules more powerful and easier to manage. The most common states are:
; <code>NEW</code>: trying to create a new connection
; <code>ESTABLISHED</code>: part of an already-existing connection
; <code>RELATED</code>: assigned to a packet that is initiating a new connection and which has been "expected"; the aforementioned mini-ALGs set up these expectations, for example, when the <code>nf_conntrack_ftp</code> module sees an [[File transfer protocol|FTP]] "<code>PASV</code>" command
; <code>INVALID</code>: the packet was found to be [[invalid packet|invalid]], e.g. it would not adhere to the [[TCP connection|TCP state]] diagram
; <code>UNTRACKED</code>: a special state that can be assigned by the administrator to bypass connection tracking for a particular packet (see raw table, above).

A normal example would be that the first packet the conntrack subsystem sees will be classified "new", the reply would be classified "established" and an [[Internet Control Message Protocol|ICMP]] error would be "related". An ICMP error packet which did not match any known connection would be "invalid".

===Connection tracking helpers===
Through the use of plugin modules, connection tracking can be given knowledge of application-layer protocols and thus understand that two or more distinct connections are "related". For example, consider the [[File Transfer Protocol|FTP]] protocol. A control connection is established, but whenever data is transferred, a separate connection is established to transfer it. When the <code>nf_conntrack_ftp</code> module is loaded, the first packet of an FTP data connection will be classified as "related" instead of "new", as it is logically part of an existing connection.

The helpers only inspect one packet at a time, so if vital information for connection tracking is split across two packets, either due to [[IP fragmentation]] or TCP segmentation, the helper will not necessarily recognize patterns and therefore not perform its operation. IP fragmentation is dealt with the connection tracking subsystem requiring defragmentation, though [[TCP segment]]ation is not handled. In case of FTP, segmentation is deemed not to happen "near" a command like <code>PASV</code> with standard segment sizes, so is not dealt with in Netfilter either.

==Network address translation==
{{Main|Network address translation}}

Each connection has a set of ''original addresses'' and ''reply addresses'', which initially start out the same. NAT in Netfilter is implemented by simply changing the reply address, and where desired, port. When packets are received, their connection tuple will also be compared against the reply address pair (and ports). Being fragment-free is also a requirement for NAT. (If need be, IPv4 packets may be refragmented by the normal, non-Netfilter, IPv4 stack.)

===NAT helpers===
Similar to connection tracking helpers, NAT helpers will do a packet inspection and substitute original addresses by reply addresses in the payload.

==Further Netfilter projects==
Though not being kernel modules that make use of Netfilter code directly, the Netfilter project hosts a few more noteworthy software.

===conntrack-tools===
<code>conntrack-tools</code> is a set of user-space tools for Linux that allow system administrators to interact with the Connection Tracking entries and tables. The package includes the <code>conntrackd</code> daemon and the command line interface <code>conntrack</code>. The userspace daemon <code>conntrackd</code> can be used to enable high availability cluster-based stateful firewalls and collect statistics of the stateful firewall use. The command line interface <code>conntrack</code> provides a more flexible interface to the connection tracking system than the obsolete <code>/proc/net/nf_conntrack</code>.

===ipset===
Unlike other extensions such as Connection Tracking, <code>ipset</code><ref>{{cite web
 | url        = http://ipset.netfilter.org/
 | title      = IP sets
 | publisher  = ipset.netfilter.org
 | access-date = 2014-07-04}}</ref> is more related to <code>iptables</code> than it is to the core Netfilter code. <code>ipset</code> does not make use of Netfilter hooks for instance, but actually provides an <code>iptables</code> module to match and do minimal modifications (set/clear) to IP sets.

The user-space tool called <code>ipset</code> is used to set up, maintain and inspect so called "IP sets" in the Linux kernel. An IP set usually contains a set of [[IP address]]es, but can also contain sets of other network numbers, depending on its "type". These sets are much more lookup-efficient than bare <code>iptables</code> rules, but of course may come with a greater memory footprint. Different storage algorithms (for the data structures in memory) are provided in <code>ipset</code> for the user to select an optimum solution.

Any entry in one set can be bound to another set, allowing for sophisticated matching operations. A set can only be removed (destroyed) if there are no <code>iptables</code> rules or other sets referring to it.

===SYN proxy===
<code>SYNPROXY</code> target makes handling of large [[SYN flood]]s possible without the large performance penalties imposed by the connection tracking in such cases. By redirecting initial <code>SYN</code> requests to the <code>SYNPROXY</code> target, connections are not registered within the connection tracking until they reach a validated final <code>ACK</code> state, freeing up connection tracking from accounting large numbers of potentially invalid connections. This way, huge <code>SYN</code> floods can be handled in an effective way.<ref>{{cite web
 | url        = https://lwn.net/Articles/563151/
 | title      = netfilter: implement netfilter SYN proxy 
 | date       = 2013-08-07
 | access-date = 2013-11-05
 | author     = Patrick McHardy
 | publisher  = [[LWN.net]]}}</ref>

On 3 November 2013, <code>SYN</code> proxy functionality was merged into the Netfilter, with the release of version 3.12 of the Linux kernel mainline.<ref>{{cite web
 | url        = https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=48b1de4c110a7afa4b85862f6c75af817db26fad
 | title      = netfilter: add SYNPROXY core/target
 | date       = 2013-08-27
 | access-date = 2013-11-05
 | publisher  = kernel.org}}</ref><ref>{{cite web
 | url        = https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4ad362282cb45bbc831a182e45637da8c5bd7aa1
 | title      = netfilter: add IPv6 SYNPROXY target
 | date       = 2013-08-27
 | access-date = 2013-11-05
 | publisher  = kernel.org}}</ref>

===ulogd===
<code>ulogd</code> is a user-space daemon to receive and log packets and event notifications from the Netfilter subsystems. <code>ip_tables</code> can deliver packets via the userspace queueing mechanism to it, and connection tracking can interact with <code>ulogd</code> to exchange further information about packets or events (such as connection teardown, NAT setup).

===Userspace libraries===
Netfilter also provides a set of libraries having <code>libnetfilter</code> as a prefix of their names, that can be used to perform different tasks from the userspace. These libraries are released under the GNU GPL version 2. Specifically, they are the following:
; <code>libnetfilter_queue</code>
: allows to perform userspace packet queueing in conjunction with iptables; based on <code>libnfnetlink</code>
; <code>libnetfilter_conntrack</code>
: allows manipulation of connection tracking entries from the userspace; based on <code>libnfnetlink</code>
; <code>libnetfilter_log</code>
: allows collection of log messages generated by iptables; based on <code>libnfnetlink</code>
; <code>libnl-3-netfilter</code>
: allows operations on queues, connection tracking and logs; part of the <code>libnl</code> project<ref>{{cite web
 | url        = https://www.infradead.org/~tgr/libnl/doc/api/group__nfnl.html
 | title      = Netfilter Library (libnl-nf)
 | date       = 2013-04-02
 | access-date = 2013-12-28
 | publisher  = infradead.org}}</ref>
; <code>libiptc</code>
: allows changes to be performed to the iptables firewall rulesets; it is not based on any <code>[[netlink]]</code> library, and its [[API]] is internally used by the <code>iptables</code> utilities
; <code>libipset</code>
: allows operations on IP sets; based on <code>libmnl</code>.

==Netfilter workshops==
The Netfilter project organizes an annual meeting for developers, which is used to discuss ongoing research and development efforts. The 2018 Netfilter workshop took place in Berlin, Germany, in June 2018.<ref>{{cite web
 | url          = https://workshop.netfilter.org/2018/|title=14th Netfilter Workshop
 | date         = 2018-09-26
 | publisher    = workshop.netfilter.org
 | access-date   = 2018-09-26}}</ref>

==See also==
{{Portal|Free and open-source software|Linux}}
{{Div col|colwidth=25em}}
* [[Berkeley Packet Filter]]
* [[Open source license litigation#Geniatech v. McHardy_(2018)|Geniatech v. McHardy (2018)]] lawsuit
* [[IP Virtual Server]] (IPVS, part of LVS)
* [[ipchains]], the predecessor to [[iptables]]
* [[ipfirewall|ipfw]]
* [[Linux Virtual Server]] (LVS)
* [[Netlink]], an API used by Netfilter extensions
* [[Network scheduler]], another low-level component of the network stack
* [[NPF (firewall)]]
* [[PF (firewall)]]
* [[Uncomplicated Firewall]]
{{div col end}}

==References==
{{Reflist|30em}}

==External links==
* {{Official website|https://netfilter.org/}}
* [https://netfilter.org/projects/conntrack-tools/ conntrack-tools homepage]
* [http://ipset.netfilter.org/ ipset homepage]
* [https://netfilter.org/projects/ulogd/ ulogd homepage]
* [https://workshop.netfilter.org/ Home of the Netfilter Workshop websites]
* "[https://inai.de/documents/Netfilter_Modules.pdf Writing Netfilter Modules]" (e-book; 2009)
* "[https://www.zdnet.com/home-and-office/networking/netfilter-and-iptables-stateful-firewalling-for-linux/ Netfilter and Iptables &mdash; Stateful Firewalling for Linux]" (11 October 2001)
* [https://web.archive.org/web/20121019131903/http://www.linuxfoundation.org/collaborate/workgroups/networking/networkoverview Network overview by Rami Rosen]
{{Firewall software}}
{{Linux kernel}}

[[Category:Firewall software]]
[[Category:Free network-related software]]
[[Category:Free security software]]
[[Category:Linux kernel features]]