26 December, 2006
Syngress Publishing has published a case study
entitled
"Active Response" I wrote for the book
Wireshark & Ethereal Network Protocol Analyzer Toolkit

(see pages 398-402).
This case study explores the usage of
Wireshark to
examine the structure of TCP RST (reset) packets that are generated by the
iptables REJECT target and by the
flexresp
and
flexresp2 Snort detection plugins in response to
malicious traffic sent against a webserver. Because each of these mechanisms employs a different
strategy for creating the RST packets, it is possible for an attacker to perform some passive
fingerprinting in an effort to discover the response mechanism. For example, iptables rules that
utilize the REJECT target (see the iptables command below) generate packets from within the Linux
kernel and hard code the TTL value at 255 for all kernel versions < 2.6.16.
# iptables -I INPUT 1 -p tcp --dport 80 -m string --string "/etc/passwd" --algo bm -j REJECT --reject-with tcp-reset
The REJECT target can only send the RST packet to the source IP that matched the REJECT rule.
The flexresp detection plugin can send RST packets to both sides of a TCP connection, always sets the
TCP window size to zero, and selects a random TTL value between 64 and 255. The remaining analysis
can be found in the book, and provides additional details on characteristics of the RST packets sent
by each response mechanism.