How Firewalls Work: Packet Filtering, Stateful Inspection, and Next-Gen Firewalls

What Is a Firewall?

A firewall is a network security device — hardware, software, or a combination — that monitors network traffic between two or more networks and enforces security policies by allowing or blocking traffic based on rules. The term comes from the physical firewall concept: a barrier designed to prevent fire from spreading from one area to another. In networking, firewalls sit at the boundary between trusted networks (such as a corporate LAN or home network) and untrusted networks (such as the internet), acting as a gatekeeper that inspects traffic and enforces security policies.

Firewalls are not a single technology but a category that spans several architectural generations, each more sophisticated than the last. Understanding how each type works explains both their strengths and their limitations.

Packet Filtering Firewalls

The earliest firewalls, appearing in the late 1980s, operated at the network layer (Layer 3) of the OSI model as packet filters. A packet filter examines each incoming and outgoing packet in isolation, comparing its header fields against a set of access control rules. Typical filter criteria include: source and destination IP addresses, source and destination ports, and the protocol (TCP, UDP, ICMP).

For example, a rule might permit inbound TCP traffic on port 443 (HTTPS) from any source address, while blocking all inbound connections on port 23 (Telnet). Packet filters are extremely fast because they process minimal information — only the packet header, not the payload — and require no memory of previous packets.

The critical limitation of packet filters is that they are stateless: they have no memory of previous packets and cannot track whether a packet is part of an established, legitimate connection or a spoofed packet attempting to bypass the firewall by mimicking the appearance of legitimate traffic. This makes them vulnerable to certain attacks, such as IP spoofing and fragmentation attacks.

Stateful Inspection Firewalls

Introduced by Check Point in 1993 with its FireWall-1 product, stateful inspection (also called dynamic packet filtering) overcomes the key limitation of stateless packet filters by tracking the state of active network connections. A stateful firewall maintains a state table — a database of all current TCP connections and UDP pseudo-connections — and uses this context when evaluating each new packet.

When a client inside the network initiates an outbound TCP connection (sending a SYN packet), the firewall records this in the state table. When the server responds with a SYN-ACK, the firewall recognizes this as a legitimate return packet for an established connection and allows it through, even if the inbound rule would otherwise block unsolicited inbound connections from that server. This makes stateful firewalls significantly more secure against spoofing and many connection-based attacks.

Stateful firewalls operate at Layers 3 and 4 of the OSI model. They understand TCP's three-way handshake and can detect anomalous connection states — such as a SYN flood attack (sending large numbers of SYN packets without completing the handshake) — and respond by dropping suspicious traffic or limiting connection rates.

Application Layer Firewalls and Proxies

Application layer firewalls operate at Layer 7 of the OSI model and can understand application-level protocols such as HTTP, FTP, DNS, and SMTP. By inspecting the actual content of traffic — not just headers — these firewalls can enforce policies based on application behavior. For example, an application layer firewall can block file transfers over HTTP while allowing web browsing, or detect and block SQL injection attempts embedded in HTTP requests.

Application proxies work by acting as intermediaries: client requests go to the proxy, the proxy makes the request on the client's behalf, and the response is returned to the client. This complete intermediation allows the proxy to perform deep content inspection and filtering, but it also adds latency and can create performance bottlenecks at scale.

Next-Generation Firewalls (NGFW)

Next-generation firewalls, a term popularized by Gartner and commercialized by Palo Alto Networks starting in 2007, integrate the capabilities of traditional stateful firewalls with deep packet inspection, application identification and control, user identity awareness, SSL/TLS inspection, and integrated threat intelligence.

The defining capability of NGFWs is application awareness: the ability to identify and control traffic based on the specific application generating it, regardless of port or protocol. This matters because modern applications frequently use non-standard ports or tunnel themselves over HTTP/HTTPS to bypass traditional port-based firewall rules. An NGFW can identify whether traffic on port 80 is a web browser, a peer-to-peer file sharing application, or a command-and-control connection — and apply different policies to each.

NGFWs also integrate intrusion prevention system (IPS) capabilities, blocking known exploit traffic and malicious signatures in real time. Many NGFWs include sandboxing — automatically detonating suspicious files in an isolated environment to observe their behavior before allowing them into the network. User identity integration (connecting firewall policies to Active Directory users rather than IP addresses) enables granular, role-based access control.

Hardware vs. Software Firewalls

Firewalls are deployed as dedicated hardware appliances, software running on general-purpose servers, or increasingly as virtual appliances in cloud environments. Hardware firewalls (such as those from Cisco, Palo Alto Networks, Fortinet, and Check Point) are optimized for high-throughput network traffic inspection and are used at network perimeters in enterprise environments. They offer predictable performance but require physical installation and maintenance.

Software firewalls include host-based firewalls installed on individual endpoints (such as Windows Defender Firewall) and software-defined network firewalls in virtualized environments. Host-based firewalls provide per-endpoint protection and are important in environments where east-west traffic (between internal systems) is a concern — particularly relevant given the rise of lateral movement attacks in which attackers who have breached the network perimeter move from system to system.

Web Application Firewalls (WAF)

A Web Application Firewall (WAF) is a specialized firewall designed to protect web applications by filtering, monitoring, and blocking HTTP/HTTPS traffic between a web application and the internet. Unlike traditional firewalls that operate primarily at the network and transport layers, WAFs operate at the application layer and understand the HTTP protocol, including request methods, headers, cookies, and form inputs.

WAFs are designed to detect and block web application attacks defined in the OWASP Top Ten, including SQL injection, cross-site scripting (XSS), cross-site request forgery (CSRF), and command injection. They can operate in two modes: detection mode (logging and alerting on suspicious traffic without blocking) and prevention mode (actively blocking identified attack traffic). WAFs are deployed as reverse proxies in front of web servers, as cloud services (such as AWS WAF or Cloudflare WAF), or as modules integrated into web servers or load balancers.