Hide Your Nodes, AAAAAA@AAAAAAA is Coming

06:04:59.473 [error] ** :"myapp@myhost": Connection attempt from node :AAAAAA@AAAAAAA rejected since it cannot handle [~c"UNLINK_ID", ~c"HANDSHAKE_23", ~c"BIG_CREATION", ~c"BIT_BINARIES", ~c"EXPORT_PTR_TAG"].**

This is NOT what you want to suddenly pop up in your logs at 10 PM.

I am a simple man. I host my silly little hobby projects on a single VPS server behind an NGINX proxy. There is nothing sensitive there, no user data; they don't even user cookies! I'm sure I'll be fine with the perfectly sensible security defaults, right?

Well, yes and no. Yes, nothing bad happened so far. No, I'm not totally fine; this log message almost gave me a heart attack.

        ┌───┬───────┬───────┬───────┬───────┬───┐
        │   │       │       │       │       │   |
        ├───┴───┬───┴───┬───┴───┬───┴───┬───┴───┤
        │       │       │       │       │       |
        ├───┬───┴───┬───┴───┬───┴───┬───┴───┬───┤
        │   │   ┌───┴───────┴───────┴───┐   │   |
        ├───┴───┤       WANTED          ├───┴───┤
        │       ├───────────────────────┤       │
        ├───┬───┤                       ├───┬───┤
        │   │   │                       │   │   │
        ├───┴───┤                       ├───┴───┤
        │       │                       │       │
        ├───┬───┤   AAAAAA@AAAAAAA      ├───┬───┤
        │   │   │                       │   │   │
        ├───┴───┤                       ├───┴───┤
        │       │                       │       │
        ├───┬───┤                       ├───┬───┤
        │   │   ├───────────────────────┤   │   │
        ├───┴───┤ FOR UNAUTHORIZED      ├───┴───┤
        │       │ ACCESS ATTEMPT        │       │
        ├───┬───┴───┬───────┬───────┬───┴───┬───┤
        │   │       │       │       │       │   │
        ├───┴───┬───┴───┬───┴───┬───┴───┬───┴───┤
        │       │       │       │       │       │
        └───────┴───────┴───────┴───────┴───────┘

Have you seen this node?

Who is :AAAAAA@AAAAAAA?

Ah yes, this is just the right amount of As to go and search for it on the internet. It isn't hard, so I'll spare you the details. Apparently there was a CouchDB exploit that allowed for an attacker to perform a remote code execution by connecting to an Erlang node. It used AAAAAA@AAAAAAA as the fake node name. And when there is an exploit, there is automation. Somebody out there running this exploit (and no doubt many others) looking for unsecured Erlang nodes to connect to. My app node rejected the connection, but the attempt got close enough to be logged and we definitely don't want that.

I am the AAAAAA@AAAAAAA now

Before we fix anything, we need to reproduce the problem. Theoretically, we can just run the Python script from the link above, but that's no fun. Let's go step by step. This will also help us implement security measures later.

First of all, epmd. The very first thing the hacker is doing is looking if there's even an Erlang node on the server. For this, they just ask epmd through the default port (4369). We can do this with netcat:

echo -ne "\x00\x01\x6e" | nc <IP> 4369
name my_app at port 38255
name another_app at port 43493

Huh, you guys aren't hiding from the world at all, are you? Anyway, now we have a list of nodes and their ports. Let's connect using the binary message from the script:

echo -ne "\x00\x15n\x00\x07\x00\x03\x49\x9cAAAAAA@AAAAAAA" | nc <IP> 38255

snot_allowed

Boom, we got it! The log message is the same, but we're the attacker now!

Hobby-grade Security Plan

Time to devise and implement our hobby-grade security plan. And I say hobby-grade for a reason. If you're truly running a distributed cluster in Kubernetes or what not, you won't find any advice here.

1. Disable Erlang Distribution?

If we're running a single node, we don't need Erlang distribution, right? Welllll. Strictly speaking, we don't, but it's required for some nice bin/myapp commands, such as remote. I personally can't give up on that, but if you can, look for the RELEASE_DISTRIBUTION environment variable in the release.

2. Disable epmd

OK, we might want to run a distribution, but can we at least not advertise our nodes to the world with epmd? Yes! The keyword here is "epmdless". Basically, you can disable automatic epmd start and hardcode a port your app will be using.

But don't be fooled, even though epmd isn't there to snitch, the port is still open and discoverable:

nmap -sV -Pn -n -T4 -p- <IP>
Starting Nmap 7.95 ( https://nmap.org ) at 2025-03-01 13:01 PST
Nmap scan report for <IP>
Host is up (0.028s latency).
Not shown: 65529 closed tcp ports (conn-refused)
PORT     STATE SERVICE        VERSION
...
6789/tcp open  ibm-db2-admin?

Nooooo, this means attackers can still attempt to connect to the node! It just got harder to do!

3. Firewall

I regret to inform you, you might actually have to configure a firewall. At least that's what I did! I can't help you much here, except for maybe a little glimpse of hope: it wasn't actually that hard to do.

Was There Ever a Threat?

Kind of. There are a couple of hoops the attacker would need to jump through in order to actually connect, like dealing with short node names and knowing the magic cookie, but Erlang documentation states that those mechanisms aren't designed to ensure security.

Conclusion

There is someone out there dressed up as an Erlang node going from server to server knocking on port 4369. You're probably safe, but the knock is creepy nonetheless. Make sure you keep your epmd under control and your ports closed, otherwise... :AAAAAA@AAAAAAA WILL GET YOU!

Alex Martsinovich