The 2.6.11 release of fwknop is available for
download (or via the
github release tag).
Here is the complete
ChangeLog:
(The primary material for this blog post was
released on github. I'm reproducing
part it here as a blog post.)
Over the long term, the Wireguard VPN is set to send shockwaves
through the VPN community with its modern cryptographic design, performance, stealthiness against
active network scanners, and commitment to security through a minimally complex code base. It is
my belief that these characteristics firmly place Wireguard among the best VPN options available.
Over time, it is likely that commercial solutions will be developed around Wireguard similarly to
commercial wrappers around OpenVPN.
This repository is dedicated to deploying a Wireguard VPN on macOS via a Linux VM running
under a virtualization solution such as Parallels. There are many alternatives to this
approach - including omitting the Linux piece altogether and using the cross-platform macOS
Wireguard tools - but I'm interested in using the
Wireguard kernel module from a Mac. This has to be done from a Linux VM or container, and we'll
talk about the VM route in this write up.
The primary use case for running such a VPN solution is to provide security for network traffic
emanating from a Mac laptop that is connected to a potentially hostile wireless network. This
includes the network at the local coffee shop among many others. Nothing against coffee shops of
course (I love coffee), but they are in the business of making wonderful caffeinated potions - not
hardening their network infrastructure against adversaries. In terms of general threats to network
traffic, a properly deployed VPN allows you to shift much of the security burden to the other side
of the VPN. In this case, the remote Wireguard end point will be deployed in a major cloud provider
or ISP network. The security of, say, Google's GCE network or Amazon's AWS network is far higher
than the network of the local coffee shop.
Note macOS security is a broad topic, and this repository is meant only to discuss a VPN
solution based on Wireguard. For a comprehensive treatment of macOS security, including other
VPN options, see the excellent
macOS Security and Privacy Guide.
The 2.6.10 release of fwknop is available for
download (or via the
github release tag).
Here is the complete
ChangeLog:
A pair of software releases is available for download -
psad-2.4.6 and fwsnort-1.6.8.
The main change is that now both pieces of software support the Snort 'metadata'
keyword. This keyword and associated field is a common fixture of modern Snort rule
sets, and usually contains important data such as IPS policy preferences, information
about vulnerable target software or OS, date created, and more.
As an example, when fwsnort detects TCP traffic over port 21 that matches the Snort rule
"ET ATTACK_RESPONSE FTP inaccessible directory access COM2" (sid 2000500), the
following syslog message is generated:
Jul 30 21:24:44 moria kernel: [650982.555939] [1] SID2000500 ESTAB IN=enx0014d1b0da65 OUT= MAC=00:12:34:56:78:65:60:e3:27:39:12:34:56:00 SRC=192.168.10.11 DST=192.168.10.1 LEN=59 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=58801 DPT=21 WINDOW=4117 RES=0x00 ACK PSH URGP=0 OPT (0101080A4538966A09B20FBC)When psad monitors this out of the syslog data, an email alert is generated as usual. However, in this email alert the metadata 'created_at' and 'updated_at' fields are now included as defined in the original rule:
"ET ATTACK_RESPONSE FTP inaccessible directory access COM2" dst port: 21 (no server bound to local port) flags: ACK PSH content: "/COM2/" content: "/COM2/" sid: 2000500 chain: FWSNORT_INPUT_ESTAB packets: 36 classtype: string-detect reference: (url) http://doc.emergingthreats.net/bin/view/Main/2000500 reference: (url) http://doc.emergingthreats.net/bin/view/Main/2000500 created_at 2010_07_30 updated_at 2010_07_30
The 2.6.9 release of fwknop is available for
download (or via the
github release tag).
Here is the complete
ChangeLog:
A major new feature in fwknop has been introduced today with the
2.6.8 release
(github tag) - the
ability to integrate with third-party devices. This brings SPA operations easily to
any device or software that offers a command line interface. By default,
the fwknop daemon supports four different firewalls: iptables, firewalld, ipfw, and PF.
But, suppose you want to have fwknopd leverage
ipset instead? Or, suppose you have an
SSH pre-shared key between a Linux system and a Cisco router, and you want fwknopd
(running on the Linux box) to control the ACL on the router for the filtering portion of
SPA? Finally, suppose that you want a stronger measure of protection for an SSH daemon
that may have been backdoored, and that runs on a
proprietary OS where fwknopd can't be deployed natively? The sky is the limit, and
I would be interested in hearing about other deployment scenarios.
These scenarios and many others are now supported with a new "command open/close cycle"
feature in fwknop-2.6.8. Essentially, fwknopd has the ability to execute an arbitrary command
upon receiving a valid SPA packet (the "open"), and then execute a different command after a
configurable timeout (the "close"). This allows fwknopd to integrate with any third-party
device or software if open and close commands can be defined for how to interact. These
commands are specified on a per-stanza basis in the access.conf file, and a set of variable
substitutions are supported such as '$SRC', '$PORT',
'$PROTO', and '$CLIENT_TIMEOUT'. Naturally, the IP address, port, and
protocol are authenticated and decrypted out a valid SPA packet - i.e., SPA packet headers
are not trusted.
Let's see an example on a Linux system ("spaserver"). Here, we're going to have fwknopd interface with
ipset instead of iptables. First, we'll create an ipset named fwknop_allow,
and we'll link it into the local iptables policy. If a packet hits the
fwknop_allow ipset and there is no matching source IP, then the DROP rule at the
end of the iptables policy implements the default-drop policy. No userspace daemon such
as SSHD can be scanned or otherwise attacked from remote IP addresses without first
producing a valid SPA packet.
[spaserver]# ipset create fwknop_allow hash:ip,port timeout 30Now, we create a stanza in the fwknop /etc/fwknop/access.conf file and fire up fwknopd like this:
[spaserver]# cat /etc/fwknop/access.conf SOURCE ANY KEY_BASE64 <base64 string> HMAC_KEY_BASE64 <base64 string> CMD_CYCLE_OPEN ipset add fwknop_allow $SRC,$PROTO:$PORT timeout $CLIENT_TIMEOUT CMD_CYCLE_CLOSE NONEWith fwknopd running and iptables configured to drop everything except for IP communications that match the fwknop_allow ipset, let's use the fwknop client from a remote system "spaclient" to gain access to SSHD on the server for 30 seconds (note that the iptables conntrack module will keep the connection open after the SPA client IP is removed from the ipset). We'll assume that the encryption and HMAC keys have been previous shared between the two systems, and on the client these keys have been written to the "spaserver" stanza in the ~/.fwknoprc file:
[spaclient]$ fwknop -A tcp/22 -a 1.1.1.1 -f 30 -n spaserverSo, behind the scenes after the SPA packet has been sent above, fwknopd on the server has authenticated and decrypted the SPA packet, and has executed the following ipset command. In this case, there is no need for a corresponding close command because ipset implements the timer provided by the client itself, so the client IP is deleted from the ipset automatically. (In other scenarios, the close command can be fully specified instead of using the string 'NONE' as we have above.) Here are the syslog messages that fwknopd has generated, along with the 'ipset list' command output to show the 1.1.1.1 IP as a member of the set:
[spaserver]# grep fwknopd /var/log/syslog |tail -n 2 Dec 23 15:38:06 ubuntu fwknopd[13537]: (stanza #1) SPA Packet from IP: 1.2.3.4 received with access source match Dec 23 15:38:06 ubuntu fwknopd[13537]: [1.2.3.4] (stanza #1) Running CMD_CYCLE_OPEN command: /sbin/ipset add fwknop_allow 1.1.1.1,6:22 timeout 30 Name: fwknop_allow Type: hash:ip,port Revision: 5 Header: family inet hashsize 1024 maxelem 65536 timeout 30 Size in memory: 224 References: 0 Members:In addition to the ability to swap out the existing firewall with a completely different filtering infrastructure, there are other notable features and fixes in the 2.6.8 release. The most important of these is a new feature implemented by Jonathan Bennett (and suggested by Hank Leininger in github issue #62) that allows access.conf files to be imported via a new '%include' directive. This can be advantageous in some scenarios by letting non-privledged users define their own encryption and authentication keys for SPA operations. This way, users do not need write permissions to the main /etc/fwknop/access.conf file to change keys around or define new ones.
The 2.6.7 release of fwknop is available for
download (or via the
github release tag).
This release adds significant support for running commands delivered by SPA packets via
'sudo' on the server, and this allows the powerful 'sudoers' syntax to filter commands
that remote users are allowed to execute.
In addition, the --key-gen (key generation) mode has been added to
fwknopd. This will allow better integration with Jonathan Bennett's
Fwknop2 Android client - particularly
when only the fwknopd server is installed on a system (as is usually the case for embedded
distributions such as OpenWRT). Further, Jonathan contributed a
console QR code generator, so that fwknop encryption and HMAC keys can be imported into
the Fwknop2 Android client via the phone's camera. Here is an example:
$ fwknopd --key-gen | ./extras/console-qr/console-qr.sh
In other news, Jonathan and I will be giving a lengthy interview on
Single Packet Authorization with fwknop for the
FLOSS Weekly show organized by the venerable
Randal Schwartz
of perl fame. Tune in September 2nd at 11am Eastern time.
As usual, fwknop has a Coverity Scan score of zero,
and the code coverage report achieved by the 2.6.7 test suite is available
here. Note that the fwknop test suite is now
achieving 90% code coverage counted by lines, and 100% code coverage counted by
functions. This reflects the commitment the fwknop project makes towards rigorous
security and testing quality.
Here is the complete
ChangeLog for fwknop-2.6.7:
Jonathan Bennett's new Android
'Fwknop2' client is ready for prime time. It is available now in the
Google Play store as well as on
F-Droid,
and Jonathan put together a nice video demonstration of Fwknop2 being used to access
SSHD on a router running OpenWRT. Also demonstrated is the usage of NAT to transparently
access SSHD running on a system behind the router. This illustrates the ability
fwknopd offers for creating inbound NAT rules for external SPA clients - something that is,
to my knowledge, unique to the fwknop in the world of SPA software. Finally, in an
innovative twist, the Fwknop2 client can read encryption and authentication keys from
a QR code via the phone's camera. This simplifies the task of getting longer keys
- particularly those that are base64-encoded - to the phone for SPA operations.
After a long while without updates to the fwknop clients on Android or iOS, I'm
excited to announce that Jonathan Bennett
has developed an entirely new client for Android called Fwknop2. This
client adds significant features such as the ability to save configurations (including
both encryption and authentication keys), proper handling of base64 encoded keys, and
support for manually specified IP addresses to be allowed through the remote firewall.
Further, Fwknop2 integrates with
JuiceSSH
so that an SSH connection can seamlessly be launched right after the SPA packet is
sent. Finally, in an interesting twist, Fwknop2 will be able to read encryption
and HMAC keys via a QR code via the camera. The QR code itself is generated from
key material in --key-gen mode (which is currently available in the standard fwknop
client and will be available in the fwknopd server in the next release).
Fwknop2 will be available on both the Google Play store and via F-Droid within
the next few hours, and in the meantime the APK is available on github
here.
Below are a couple of screenshots of the new Android client in action - these are
from an Android VM running under Parallels on Mac OS X (Yosemite):
People who use Single Packet Authorization (SPA) or its security-challenged cousin
Port Knocking (PK) usually access SSHD running on the same system where the SPA/PK
software is deployed. That is, a firewall running on a host has a default-drop
policy against all incoming SSH connections so that SSHD cannot be scanned, but a
SPA daemon reconfigures the firewall to temporarily grant access to a passively
authenticated SPA client:
This works
well enough,
but both port knocking and SPA work in conjunction with a firewall, and "important" firewalls
are usually gateways between networks as opposed to just being deployed on standalone hosts.
Further, Network Address Translation (NAT) is commonly used on such firewalls (at least
for IPv4 communications) to provide Internet access to internal networks that are on
RFC 1918 address space, and also to allow external hosts access to services hosted on
internal systems.
The prevalence of NAT suggests that SPA should integrate strongly with it. Properly done,
this would extend the notion of SPA beyond just supporting the basic feature of granting
access to a local service. To drive this point home, and to see what a NAT-enabled vision
for SPA would look like, consider the following two scenarios:
[firewall]# cat /etc/fwknop/access.conf SOURCE ANY KEY_BASE64 wzNP62oPPgEc+k...XDPQLHPuNbYUTPP+QrErNDmg= HMAC_KEY_BASE64 d6F/uWTZmjqYor...eT7K0G5B2W9CDn6pAqqg6Oek= FW_ACCESS_TIMEOUT 1000 FORCE_SNAT 123.1.2.3 DISABLE_DNAT Y FORWARD_ALL Y [firewall]# cat /etc/fwknop/fwknopd.conf ENABLE_IPT_FORWARDING Y; ENABLE_IPT_SNAT Y; [firewall]# fwknopd -i eth0 -f Starting fwknopd Added jump rule from chain: FORWARD to chain: FWKNOP_FORWARD Added jump rule from chain: POSTROUTING to chain: FWKNOP_POSTROUTING iptables 'comment' match is available Sniffing interface: eth3 PCAP filter is: 'udp port 62201' Starting fwknopd main event loop.With the fwknopd daemon up and running on the firewall/SPA gateway system, we now run the client to gain access to the Internet after showing the "[internet]" stanza in the ~/.fwknoprc file:
[spaclient]$ cat ~/.fwknoprc [internet] KEY_BASE64 wzNP62oPPgEc+k...XDPQLHPuNbYUTPP+QrErNDmg= HMAC_KEY_BASE64 d6F/uWTZmjqYor...eT7K0G5B2W9CDn6pAqqg6Oek= ACCESS tcp/22 ### ignored by FORWARD_ALL SPA_SERVER 192.168.10.1 USE_HMAC Y ALLOW_IP source [spaclient]$ nc -v google.com 80 ### nothing comes back here because the SPA packet hasn't been sent yet ### and therefore everything is blocked (except for DNS in this case) [spaclient]$ fwknop -n internet [spaclient]$ nc -v google.com 80 Connection to google.com 80 port [tcp/http] succeeded!Back on the server, below are the rules that are added to grant Internet access to the spaclient system. Note that all ports/protocols are accepted for forwarding through the firewall, and an SNAT rule has been applied to the spaclient source IP of 192.168.10.55:
(stanza #1) SPA Packet from IP: 192.168.10.55 received with access source match Added FORWARD ALL rule to FWKNOP_FORWARD for 192.168.10.55 -> 0.0.0.0/0 */*, expires at 1429387561 Added SNAT ALL rule to FWKNOP_POSTROUTING for 192.168.10.55 -> 0.0.0.0/0 */*, expires at 1429387561Conclusion
Last week there was an interesting question
posed by Peter Smith
to the fwknop mailing list that focused on running fwknop in
UDP listener mode
vs. using libpcap. I thought it would be useful to turn this into a blog post, so here is Peter's
original question along with my response:
"...I want to deploy fwknop on my server, but I'm not sure If I should use
the UDP listener mode or libpcap. At first UDP listener mode seems to be
the choice, because I don't have to compile libpcap. However, I then
have to open a port in the firewall. Thinking about this, I get the
feeling that I'm defeating the purpose of using SPA, by allowing
Internet access to a privileged processe.
If an exploitable security issue is found, even though fwknop remains
passive and undiscoverable, an attacker could blindly send his exploit
to random ports on servers he suspects running fwknopd, and after
maximum 65535 tries he would have root access. I'm not a programmer, so
I can't review the code of fwknop or SSH daemon, but if both is equally
likely of having security issues, I might as well just allow direct
access to the SSH daemon and skip using SPA.
Is my point correct?..."
First, let me acknowledge your point as an important issue to bring up. If
someone finds a vulnerability in fwknopd, it doesn't matter whether fwknopd
is collecting packets via UDP sockets or from libpcap (assuming we're
talking about a vulnerability in fwknopd itself). The mere processing of
network traffic is the problem.
So, why run fwknopd at all?
This is something of a long-winded answer, but I want to be thorough. I'll
likely not settle the issue with this response, but it's good to start the
discussion.
Starting with your example above with SSHD open to the world, suppose there
is a vulnerability in SSHD. What does the exploit model look like? Well, an
attacker armed with an exploit can trivially find the SSH daemon with any
scanner, and from there a single TCP connection is sufficient for
exploitation. I would argue that a primary enabling factor benefiting the
attacker in this scenario is that vulnerable targets that listen on TCP
sockets are so easy to find. The combination of scannable services +
zero-day exploits is just too nice for an attacker. Several years ago I
personally had a system running SSHD compromised because of this. (At the
time, it was my fault for not patching SSHD before I put it out on the
Internet, but still - the system was compromised within about 12 hours of
being online.)
Now, in the fwknopd world, assuming a vulnerability, what would
exploitation look like? Well, targets aren't scannable as you say, so an
attacker would have to guess that a target is running fwknopd and the
corresponding port on which it watches/listens for SPA packets [1].
Forcing an attacker to send thousands of packets to different ports (assuming a
custom non-default port instead of UDP port 62201 that fwknopd normally
watches) is likely a security benefit in contrast to an obviously
targetable service. That is, sending tons of SPA exploit packets to
different ports is a common pattern that is quite noisy and is frequently
flagged by IDS's, firewalls, SIEM's, and flow analysis engines. How many
systems could an attacker target with such heavyweight scans before the
attacker's own ISP would notice? Or before the attacker's IP is included
within one of the Emerging Threats compromised hosts lists? Or within
DShield as a known bad actor? 10 million? How many of these are actually
running fwknopd? I know there are some spectacular scanning results out
there, so it's really hard to quantify this, but either way there is a big
difference between sending thousands of > 100-byte UDP packets to each
target vs. a port sweep for TCP/22.
Further, when a system is literally blocking every incoming packet [2],
an attacker can't even necessarily be sure there is any target connected to
the network at all. Many attackers would likely move on fairly quickly from
a system that is simply returning zero data to some other system.
In contrast, for a service that advertises itself via TCP, an attacker
immediately knows - with 100% certainty - there is a userspace daemon with
a set of functions that they can interact with. This presents a risk. In my
view, this risk is greater than the risk of running fwknopd where an
attacker gets no data.
Another fair question to ask is about the architecture of fwknopd. When an
HMAC is used (this will be the default in a future release), fwknopd reads
SPA packet data, computes an HMAC, and does nothing else if the HMAC does
not match. This is an effort to try to stay faithful to a simplistic
design, and to minimize the functions an attacker can interact with - even
for packets that are blindly sent to the correct port where fwknopd is
watching. Thus, most functions in fwknopd, including those that parse
user-supplied data and hence where bugs are more likely to crop up, are not
even accessible without first passing the HMAC which is a cryptographically
strong test. When libpcap is also eliminated through the use of UDP, not
even libpcap functions are in the mix [3]. In other words,
the security of fwknop does not rely on not being discoverable or scannable - it is merely
a nice side effect of not using TCP to gather data from the network.
To me, another way to think of fwknopd in UDP mode is that it provides a
lightweight generic UDP authenticator for TCP-based services. Suppose that
SSHD had a design built-in where it would bind to a UDP socket, wait for a
similar authentication packet as SPA provides, and then switch over to TCP.
In this case, there would not be a need for SPA because you would get the
best of both worlds - non-scannability [4] plus everything that TCP provides
for reliable data delivery at the same time. SPA in UDP listening mode is
an effort to bridge these worlds. Further, there is nothing that ties
fwknopd to SSH. I know people who expose RDP to the Internet for example.
Personally, I'm quite confident there are fewer vulnerabilities in fwknopd
than something like RDP. Even if there isn't a benefit in terms of the
arguments above in terms of service concealment and forcing an attacker to
be noisy, my bet is that fwknopd - even if it advertised itself via TCP -
would provide better security than RDP or other services derived from
massive code bases.
Now, beyond all of the above, there are some additional blog posts and other
material that may interest some readers:
The American Fuzzy Lop fuzzer has become a critical
tool for finding security vulnerabilities in all sorts of software. It has the ability to send fuzzing
data through programs on the order of hundreds of millions of executions per day on a capable system,
and can certainly put strain on your hardware and OS. If you are fuzzing a target program with the AFL
mode where a file is written and the target binary reads from this file, then AFL is going to conduct
a huge number of writes to the local disk. For a solid-state drive this can reduce its life expectancy
because of write amplification.
My main workstation is a Mac running OS X Yosemite, and I run a lot of Linux, FreeBSD, and OpenBSD
virtual machines under Parallels for development purposes. The drive on this system is an SSD which
keeps everything quite fast, but I don't want to prematurely shorten its life through huge AFL fuzzing
runs. Normally, I'm using AFL to fuzz fwknop from an Ubuntu-14.10 VM, and what is
needed is a way to keep disk writes down. The solution is to use a RAM disk from within the VM.
First, from the Ubuntu VM, let's get set up for AFL fuzzing and show what the disk writes look like
without using a RAM disk from the perspective of the host OS X system. This assumes AFL 0.89 has been
installed already and is in the current path:
$ git clone https://github.com/mrash/fwknop.git fwknop.git $ cd fwknop.git $ ./autogen.sh $ cd test/afl/ $ ./compile/afl-compile.shWe're not running AFL yet. Now, from the Mac, launch the Activity Monitor (under Applications > Utilities) and look at current disk utilization: So, not terrible - currently 31 writes per second at the time the snapshot was taken, and that includes OS X itself and the VM at the same time. But, now let's fire up AFL using the digest cache wrapper on the Ubuntu VM (the main AFL text UI is not shown for brevity):
$ ./fuzzing-wrappers/server-digest-cache.sh [+] All right - fork server is up. [+] All test cases processed. [+] Here are some useful stats: Test case count : 1 favored, 0 variable, 1 total Bitmap range : 727 to 727 bits (average: 727.00 bits) Exec timing : 477 to 477 us (average: 477 us) [+] All set and ready to roll!And now let's take a look at disk writes again from OS X: Whoa, that's a massive difference - nearly two orders of magnitude. AFL has caused disk writes to spike to over 2,700 per second with total data written averaging at 19.5MB/sec. Long term fuzzing at this level of disk writes would clearly present a problem for the SSD - AFL frequently needs to be left running for days on end in order to be thorough. So, let's switch everything over to use a RAM disk on the Ubuntu VM instead and see if that reduces disk writes:
# mkdir /tmp/afl-ramdisk && chmod 777 /tmp/afl-ramdisk # mount -t tmpfs -o size=512M tmpfs /tmp/afl-ramdisk $ mv fwknop.git /tmp/afl-ramdisk $ cd /tmp/afl-ramdisk/fwknop.git/test/afl/ $ ./fuzzing-wrappers/server-digest-cache.shHere is disk utilization once again from the Mac: We're back to less than 10 writes per second to the SSD even though AFL is going strong on the Ubuntu VM (not shown). The writes for the previous fuzzing run are still shown to the left of the graph (since they haven't quite aged out yet when the screenshot was taken), and new writes are so low they don't even make it above the X-axis at this scale. Although the total execs per second - about 2,000 - achieved by AFL is not appreciably faster under the RAM disk, the main benefit is that my SSD will last a lot longer. For those that don't run AFL underneath a VM, a similar strategy should still apply on the main OS. Assuming enough RAM is available for whatever software you want to fuzz, just create a RAM disk and run everything from it and extend the life of your hard drive in the process.
Over the past few months, the American Fuzzy Lop
(AFL) fuzzer written by Michal Zalewski has become a tour de force in the security field. Many interesting bugs
have been found, including a late breaking bug in the venerable cpio utility
that Michal announced to the full-disclosure list.
It is clear that AFL is succeeding perhaps where other fuzzers have either failed or not been applied, and this
is an important development for anyone trying to maintain a secure code base. That is, the ease of use coupled with
powerful fuzzing strategies offered by AFL mean that open source projects should integrate it directly
into their testing systems. This has been done for the fwknop project with some basic
scripting and one patch
to the fwknop code base. The patch was necessary because according to AFL documentation,
projects that leverage things like encryption and cryptographic signatures are not well
suited to brute force fuzzing coverage, and fwknop definitely fits into this category.
Crypto being a fuzzing barrier is not unique to AFL - other fuzzers have the same problem.
So, similarly to the libpng-nocrc.patch included in the AFL sources, encryption,
digests, and base64 encoding are bypassed when fwknop is compiled with
--enable-afl-fuzzing on the configure command line. One doesn't need to apply
the patch manually - it is built directly into fwknop as of the 2.6.4 release. This is
in support of a major goal for the fwknop project which is comprehensive
testing.
If you maintain an open source project that involves crypto and does any sort of
encoding/decoding/parsing gated by crypto operations, then you should patch your project so that
it natively supports fuzzing with AFL through an optional compile time step. As an example, here is a
portion of the patch to fwknop that disables base64 encoding by just copying data manually:
index b8423c2..5b51121 100644 int b64_decode(const char *in, unsigned char *out) { unsigned char *dst = out; v = 0; for (i = 0; in[i] && in[i] != '='; i++) { unsigned int index= in[i]-43; @@ -68,6 +80,7 @@ b64_decode(const char *in, unsigned char *out) if (i & 3) *dst++ = v >> (6 - 2 * (i & 3)); } *dst = '\0';Fortunately, so far all fuzzing runs with AFL have turned up zero issues (crashes or hangs) with fwknop, but the testing continues.
$ cd fwknop.git/test/afl/ $ ./compile/afl-compile.sh $ ./fuzzing-wrappers/spa-pkts.sh + ./fuzzing-wrappers/helpers/fwknopd-stdin-test.sh + SPA_PKT=1716411011200157:root:1397329899:2.0.1:1:127.0.0.2,tcp/22:AAAAA + LD_LIBRARY_PATH=../../lib/.libs ../../server/.libs/fwknopd -c ../conf/default_fwknopd.conf -a ../conf/default_access.conf -A -f -t Warning: REQUIRE_SOURCE_ADDRESS not enabled for access stanza source: 'ANY'+ echo -n 1716411011200157:root:1397329899:2.0.1:1:127.0.0.2,tcp/22:AAAAA SPA Field Values: ================= Random Value: 1716411011200157 Username: root Timestamp: 1397329899 FKO Version: 2.0.1 Message Type: 1 (Access msg) Message String: 127.0.0.2,tcp/22 Nat Access: <NULL> Server Auth: <NULL> Client Timeout: 0 Digest Type: 3 (SHA256) HMAC Type: 0 (None) Encryption Type: 1 (Rijndael) Encryption Mode: 2 (CBC) Encoded Data: 1716411011200157:root:1397329899:2.0.1:1:127.0.0.2,tcp/22 SPA Data Digest: AAAAA HMAC: <NULL> Final SPA Data: 200157:root:1397329899:2.0.1:1:127.0.0.2,tcp/22:AAAAA SPA packet decode: Success + exit 0 + LD_LIBRARY_PATH=../../lib/.libs afl-fuzz -T fwknopd,SPA,encode/decode,00ffe19 -t 1000 -i test-cases/spa-pkts -o fuzzing-output/spa-pkts.out ../../server/.libs/fwknopd -c ../conf/default_fwknopd.conf -a ../conf/default_access.conf -A -f -t afl-fuzz 0.66b (Nov 23 2014 22:29:48) by <lcamtuf@google.com> [+] You have 1 CPU cores and 2 runnable tasks (utilization: 200%). [*] Checking core_pattern... [*] Setting up output directories... [+] Output directory exists but deemed OK to reuse. [*] Deleting old session data... [+] Output dir cleanup successful. [*] Scanning 'test-cases/spa-pkts'... [*] Creating hard links for all input files... [*] Validating target binary... [*] Attempting dry run with 'id:000000,orig:spa.start'... [*] Spinning up the fork server... [+] All right - fork server is up.And now AFL is up and running (note the git --abbrev-commit tag integrated into the text banner to make clear which code is being fuzzed): If the fuzzing run is stopped by hitting Ctrl-C, it can always be resumed as follows:
$ ./fuzzing-wrappers/spa-pkts.sh resumeAlthough major areas in fwknop where data is consumed are effectively fuzzed with AFL, there is always room for improvement. With the wrapper scripts in place, it is easy to add new support for fuzzing other functionality in fwknop. For example, the digest cache file (used by fwknopd to track SHA-256 digests of previously seen SPA packets) is a good candidate.
The 2.6.4 release of fwknop is available for
download. New functionality has been developed
for 2.6.4, including a new UDP listener mode to remove libpcap as a dependency for
fwknopd, support for firewalld on recent versions of Fedora, RHEL, and Centos
(contributed by Gerry Reno), and support for Michal Zalewski's 'American Fuzzy Lop'
fuzzer. Further, on systems where execvpe() is available, all system() and popen()
calls have been replaced so that the shell is not invoked and no environment is used.
As usual, fwknop has a Coverity Scan score of zero,
and the code coverage report achieved by the 2.6.4 test suite is available
here.
Here is the complete ChangeLog for fwknop-2.6.4:
In this blog post I'll pose a few challenges to open source projects. These challenges,
among other goals, are designed to push automated test suites to maximize code coverage.
For motivation, I'll present code coverage stats from the OpenSSL and OpenSSH test suites,
and contrast this with what I'm trying to achieve in the fwknop project.
Stated bluntly, if comprehensive code coverage is not achieved by your test suite,
you're doing it wrong.
With major bugs like
Heartbleed and
"goto fail",
there is clearly a renewed need for better automated testing. As Mike Bland
demonstrated, the
problem isn't so much about technology since unit tests can be built for both Heartbleed
and "goto fail". Rather, the problem is that the whole software engineering life cycle
needs to embrace better testing.
Hit | Total | Coverage | ||
OpenSSL-1.0.1h | Lines: | 44604 | 96783 | 46.1 % |
Full code coverage report | Functions: | 3518 | 6574 | 53.5 % |
Branches: | 22653 | 67181 | 33.7 % | |
OpenSSH-6.6p1 | Lines: | 18241 | 33466 | 54.5 % |
Full code coverage report | Functions: | 1217 | 1752 | 69.5 % |
Branches: | 9362 | 22674 | 41.3 % | |
fwknop-2.6.3 | Lines: | 6971 | 7748 | 90.0 % |
Full code coverage report | Functions: | 376 | 377 | 99.7 % |
Branches: | 4176 | 5239 | 79.7 % |
mkdir $LCOV_DIR make clean make make tests lcov --capture --directory . --output-file $LCOV_FILE lcov -r $LCOV_FILE /usr/include/\* --output-file $LCOV_DIR/$LCOV_FILTERED genhtml $LCOV_DIR/$LCOV_FILTERED --output-directory $LCOV_DIR
clean_up();Some projects elect to write a "safe_malloc()" wrapper for malloc() or other libc functions so that error handling can be done in one place, but it is not feasible to do this for every libc function. So, how to verify whether error conditions are properly handled at run time? For malloc(), NULL is typically returned under extremely high memory pressure, so it is hard to trigger this condition and still have a functioning system let alone a functioning test suite. In other words, in the example above, how can the test suite achieve code coverage for the clean_up() function? Other examples include filesystem or network function errors that are returned when disks fill up, or a network communication is blocked, etc.
fko_get_username(fko_ctx_t ctx, char **username) fiu_return_on("fko_get_username_init", FKO_ERROR_CTX_NOT_INITIALIZED);With the fault set (there is a special command line argument --fault-injection-tag on the fwknopd server command line to enable the fault), the error handling code seen at the end of the example below is executed via the test suite. For proof of error handling execution, see the full coverage report (look at line 240).
get_spa_data_fields(fko_ctx_t ctx, spa_data_t *spdat) res = fko_get_username(ctx, &(spdat->username));
The 2.6.3 release of fwknop is available for
download. The emphasis in this release is maximizing
code coverage through a new python SPA packet fuzzer, and also on fault injection testing
with the excellent fault injection library libfiu
developed by Alberto Bertogli. Another important change in 2.6.3 is all IP resolution
lookups in '-R' mode now happen over SSL to make it harder for an adversary to mount a MITM
attack on the resolution lookup. As always, manually specifying the IP to allow through the
remote firewall is safer than relying on any network communication - even when SSL would be
involved.
Here is the complete ChangeLog for fwknop-2.6.3:
GPG_FINGERPRINT_ID 00CC95F05BC146B6AC4038C9E36F443C6A3FAD56
==11181== Conditional jump or move depends on uninitialised value(s) ==11181== at 0x113B6D: incoming_spa (incoming_spa.c:707) ==11181== by 0x11559F: process_packet (process_packet.c:211) ==11181== by 0x5270857: ??? (in /usr/lib/x86_64-linux-gnu/libpcap.so.1.4.0) ==11181== by 0x114BCC: pcap_capture (pcap_capture.c:270) ==11181== by 0x10F32C: main (fwknopd.c:195) ==11181== Uninitialised value was created by a stack allocation ==11181== at 0x113476: incoming_spa (incoming_spa.c:294)
The 2.6.0 release of fwknop is available for
download. This release incorporates a number of
feature enhancements such as an AppArmor policy for fwknopd, HMAC authenticated encryption
support for the Android client, new NAT criteria that are independently configurable
for each access.conf stanza, and more rigorous valgrind verification powered by the
CPAN Test::Valgrind module. A few bugs were fixed as well, and similarly to the 2.5 and
2.5.1 releases, the fwknop project has a Coverity defect count of zero. As proof of this,
you can see the Coverity high-level defect stats for fwknop here (you'll need to sign up
for an account):
I would encourage any open source project that is using Coverity to publish their
scan results. At last count, it appears that over 1,100 projects are using Coverity,
but OpenSSH is still not one of them.
Development on fwknop-2.6.1 will begin shortly, and here is the complete ChangeLog for
fwknop-2.6.0:
The fwknop project consistently uses valgrind to ensure that memory leaks, double free() conditions, and other problems do not creep into the code base. A high level of automation is built around valgrind usage with the fwknop test suite, and a recent addition extends this even further by using the excellent CPAN Test::Valgrind module. Even though the test suite has had the ability to run tests through valgrind, previous to this change these tests only applied to the fwknop C binaries when executed directly by the test suite. Further, some of the most rigorous testing is done through the usage of the perl FKO extension to fuzz libfko functions, so without the Test::Valgrind module these tests also could not take advantage of valgrind support. Now that the test suite supports Test::Valgrind (and a check is done to see if it is installed), all fuzzing tests can also be validated with valgrind. Technically, the fuzzing tests have been added as FKO built-in tests in the t/ directory, and the test suite runs them through Test::Valgrind like this:
# prove --exec 'perl -Iblib/lib -Iblib/arch -MTest::Valgrind' t/*.tHere is a complete example - first, run the test suite like so:
# ./test-fwknop.pl --enable-all --include perl --test-limit 3 [+] Starting the fwknop test suite... args: --enable-all --include perl --test-limit 3 [+] Total test buckets to execute: 3 [perl FKO module] [compile/install] to: ./FKO...................pass (1) [perl FKO module] [make test] run built-in tests................pass (2) [perl FKO module] [prove t/*.t] Test::Valgrind..................pass (3) [valgrind output] [flagged functions] ..........................pass (4) Run time: 1.27 minutes [+] 0/0/0 OpenSSL tests passed/failed/executed [+] 0/0/0 OpenSSL HMAC tests passed/failed/executed [+] 4/0/4 test buckets passed/failed/executedNote that all tests passed as shown above. This indicates that the test suite has not found any memory leaks through the fuzzing tests run via Test::Valgrind. But, let's validate this by artificially introducing a memory leak and see if the test suite can automatically catch it. For example, here is a patch that forces a memory leak in the validate_access_msg() libfko function. This function ensures that the shape of the access request conforms to something fwknop expects like "1.2.3.4,tcp/22". The memory leak happens because a new buffer is allocated from the heap but is never free()'d before returning from the function (obviously this patch is for illustration and testing purposes only):
$ git diff diff --git a/lib/fko_message.c b/lib/fko_message.c index fa6803b..c04e035 100644 --- a/lib/fko_message.c +++ b/lib/fko_message.c @@ -251,6 +251,13 @@ validate_access_msg(const char *msg) const char *ndx; int res = FKO_SUCCESS; int startlen = strnlen(msg, MAX_SPA_MESSAGE_SIZE); + char *leak = NULL; + + leak = malloc(100); + leak[0] = 'a'; + leak[1] = 'a'; + leak[2] = '\0'; + printf("LEAK: %s\n", leak); if(startlen == MAX_SPA_MESSAGE_SIZE) return(FKO_ERROR_INVALID_DATA_MESSAGE_ACCESS_MISSING);Now recompile fwknop and run the test suite again, after applying the patch (recompilation output is not shown):
# cd ../ # make # test # ./test-fwknop.pl --enable-all --include perl --test-limit 3 [+] Starting the fwknop test suite... args: --enable-all --include perl --test-limit 3 Saved results from previous run to: output.last/ Valgrind mode enabled, will import previous coverage from: output.last/valgrind-coverage/ [+] Total test buckets to execute: 3 [perl FKO module] [compile/install] to: ./FKO...................pass (1) [perl FKO module] [make test] run built-in tests................pass (2) [perl FKO module] [prove t/*.t] Test::Valgrind..................fail (3) [valgrind output] [flagged functions] ..........................fail (4) Run time: 1.27 minutes [+] 0/0/0 OpenSSL tests passed/failed/executed [+] 0/0/0 OpenSSL HMAC tests passed/failed/executed [+] 2/2/4 test buckets passed/failed/executedThis time two tests fail. The first is the test that runs the perl FKO module built-in tests under Test::Valgrind, and the second is the "flagged functions" test which compares test suite output looking for new functions that valgrind has flagged vs. the previous test suite execution. By looking at the output file of the "flagged functions" test it is easy to see the offending function where the new memory leak exists. This provides an easy, automated way of memory leak detection that is driven by perl FKO fuzzing tests.
# cat output/4.test [+] fwknop client functions (with call line numbers): 10 : validate_access_msg [fko_message.c:256] 6 : fko_set_spa_message [fko_message.c:184] 4 : fko_new_with_data [fko_funcs.c:263] 4 : fko_decrypt_spa_data [fko_encryption.c:264] 4 : fko_decode_spa_data [fko_decode.c:350]Currently, there are no known memory leaks in the fwknop code, and automation built around the Test::Valgrind module will help keep it that way.
(Update 10/20/2013: There is a Reddit comment thread going on this post
here.)
It has been a decade since Port Knocking was first introduced to the
security community in 2003, so it seemed fitting to recap how far the concept has evolved. Much effort has gone
into solving architectural problems with PK systems, and today
Single Packet Authorization (SPA) embodies the primary benefits
of PK while fixing its limitations.
There are noted security researchers on both sides of the debate as to whether
PK/SPA has any security value, but it is interesting that researchers who don't find value seem to concentrate on
aspects of PK/SPA that have little to do with the chief benefit: cryptographically strong concealment.
At least, this is the property offered by Single Packet Authorization but admittedly not necessarily with
Port Knocking. Let's first go through some of the more common criticisms of PK/SPA, and show what the SPA answer
is to each one. For those that haven't considered SPA in the past, perhaps it is time to give it a second
look if for no other reason than to propose a method for breaking it.
After Errata Security scanned port 22 across the entire Internet earlier this month, I thought I would go back through my iptables logs to see how the scan appeared against one of my systems. Errata Security published the IP they used for the scan as 71.6.151.167 so that it is easy to differentiate their scan from all of the other scans and probes:
[minastirith]# grep 71.6.151.167 /var/log/syslog | grep "DPT=22 " Sep 12 21:19:15 minastirith kernel: [555953.034807] DROP IN=eth0 OUT= MAC=00:13:46:11:11:11:78:cd:11:6b:11:7e:11:00 SRC=71.6.151.167 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x20 TTL=241 ID=17466 PROTO=TCP SPT=61000 DPT=22 WINDOW=0 RES=0x00 SYN URGP=0Interestingly, the SYN packet that produced the log message above does not contain TCP options. The LOG rule in the iptables policy was built with the --log-tcp-options switch, and yet the OPT field for TCP options is not included. Looking through the Masscan sources, TCP SYN packets are created with the tcp_create_packet() function which does not appear to include code to set TCP options, and neither does the default template used for describing TCP packets. This is most likely done in order to maximize performance - not from the perspective of the sender since a static hard-coded TLV encoded buffer would have done nicely - but rather to minimize the time that scanned TCP stacks must spend processing the incoming SYN packets before a response is made. While this processing time is trivial for individual TCP connections, it would start to become substantial when trying to rapidly scan the entire IPv4 address space.
### TCP connect() SYN: Sep 29 21:16:00 minastirith kernel: [171470.436701] DROP IN=eth0 OUT= MAC=00:13:46:11:11:11:78:cd:11:6b:11:7e:11:00 SRC=2.2.2.2 DST=1.2.3.4 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=15593 DF PROTO=TCP SPT=58884 DPT=12345 WINDOW=14600 RES=0x00 SYN URGP=0 OPT (020405B40402080A0CE97C070000000001030306) ### Nmap SYN (-sS): Sep 29 21:16:12 minastirith kernel: [171482.094163] DROP IN=eth0 OUT= MAC=00:13:46:11:11:11:78:cd:11:6b:11:7e:11:00 SRC=2.2.2.2 DST=1.2.3.4 LEN=44 TOS=0x00 PREC=0x00 TTL=39 ID=26480 PROTO=TCP SPT=48271 DPT=12345 WINDOW=4096 RES=0x00 SYN URGP=0 OPT (020405B4) ### Scapy SYN via: sr1(IP(dst="1.2.3.4")/TCP(dport=12345,flags="S")) Sep 29 21:35:15 minastirith kernel: [172625.207745] DROP IN=eth0 OUT= MAC=00:13:46:11:11:11:78:cd:11:6b:11:7e:11:00 SRC=2.2.2.2 DST=1.2.3.4 LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=1 PROTO=TCP SPT=20 DPT=12345 WINDOW=8192 RES=0x00 SYN URGP=0As a result, we can infer that SYN scans without options may originate from Masscan (the Scapy example not withstanding since Scapy's usage as a port scanner is a probably a lot less common than Masscan usage), and this will become more likely true over time as Masscan's popularity increases. (It is included as a tool leveraged by Rapid7's Sonar project for example.)