Like quite a few of the world’s IT/development staff, Friday 10 December 2021 didn’t go as planned for me. That’s because, late on the previous evening, the internet lit up with CVE-2021-44228.
This is a nasty remote code execution hole in a very very widely used logging component.
When the world’s biggest IT company (Amazon) still appear to have exploitable services running on the internet over 24 hours past the disclosure, well, you have to wonder how us mortals have a chance.
Cloudflare have stepped up and deployed tactics to block exploits, even for free tier customers like me - this is one of the upsides of so many people relying on a centralized giant, and reinforces my rule of thumb about avoiding running services open to the raw internet without someone cleverer than you standing in the middle.
I’m not going to talk about work here, but I am going to talk about something else which occurred to me in the aftermath of this issue.
Roughly speaking, the exploit goes like this:
- Attacker provides some untrusted input e.g.
- Untrusted input is logged by the app and executed by log4j
- Log4j fetches payload from attacker-controlled endpoint on
hostnameand executes it
- Game over
A bit of defence in depth could stop this attack at step 3. After all, when was the last time 99% of internet-connected systems needed to talk to an LDAP server on some random IP address? Or indeed any sort of LDAP server?
I thought for a brief moment that simply blocking port 389 (the default for LDAP) for outbound connections would be a cheap and simple fix.
However, you can put a port number in an LDAP URL (
so this alone isn’t the answer. But if it differentiates your server from
99% of others out there, that alone could be enough to keep you off the
radar of the mass-production exploits already being rolled out. Or even
just buy you a few extra hours to patch this sort of thing.
So, after years of putting it off, I decided to tackle the hard problem of locking down my firewalls to restrict outgoing connections.
Doing this on an internet-facing server wasn’t too painful. You can have a
look at the output of
netstat to get some idea of what’s talking outbound,
and go through the list of processes.
Building a whitelist of hosts that the system can connect to is quite hard, especially since your firewall probably talks in terms of IP addresses, and the addresses for a given hostname can change frequently.
So I started with locking down the port numbers to HTTP/S, SMTP/S, NTP, DNS and, er, that’s about it. So far nothing appears to have broken.
In due course I’ll revisit this as there are some protocols (like NTP) where locking it down further to specific hosts should be easy.
This, my friends, is much more interesting. Like most people in 2021, my house has a fair number of internet-connected devices, many of which are black boxes to me (smart thermostat, Alexa, …).
I fired up the web interface of my Mikrotik router and had a poke in the “connections” tab. I was expecting to find 99% port 443 and a bit of DNS on 53. There were a couple of surprises - Steam chatting on port 27035 and Spotify on 4070. Oh, and don’t forget to find out what port your work VPN uses!
This looks tricky enough that it will be the subject of a separate post another time.
Still, security is done in layers - can’t hurt to have one more.