Tales from the Trenches: The Other F-word

Today’s Tales from the Trenches is brought to you by Bill Burge, Senior Security Solutions Consultant.

When you’re first diving into network modeling with RedSeal, one of the initial tasks is connecting to network devices to gather their configurations. It’s a step that seems simple enough, but that elusive F-word, “Failed” becomes an all-too-familiar sight. But here’s the thing—each network has its own set of requirements, and once you crack the code, things move pretty smoothly.

That is, unless one device (or a whole set of them) decides to be a rebel. Suddenly, you’re stuck with the F-word. But wait—could that “F” word actually stand for something else? Could it be “Finding” instead of “Failed”? Maybe, just maybe, while you’re trying to pull configurations to find potential network issues, you’ve already stumbled upon something valuable about the network itself.

Customer 1: The Vegas Shuffle

Imagine this: a major Las Vegas resort and casino is trying to connect a seemingly simple firewall to a T1 for their “deal of the day” promotions. Sounds straightforward, right? Well, don’t get too excited yet. Despite repeated efforts, all they get is that dreaded “Failed.” The firewall team insists it’s up and running, and they’re logged in. Still, nothing but failure.

A bit of digging reveals the issue—turns out the IP address we’re trying to connect to is on the inside transit network of the firewall. The same subnet is defined as the failover link between the two core routers. When asked, NetOps (with a few “C” titles sprinkled in) were asked what would happen if one core router failed. Their response? The entire internal data traffic would reroute to the T1 link, leading to the “deal of the day” server. Suddenly, “Failed” isn’t just a failure—it’s a crucial finding that was previously unknown to the team.

Customer 2: The European Firewall Fiasco

Now let’s talk about a hardware and software manufacturer with a global presence. They’ve got firewalls scattered worldwide, and I’m given a list of firewalls along with a TACACS credential that’s “good for every firewall in the network.” Sounds too good to be true, doesn’t it?

After creating a couple thousand data collection tasks, everything seems to be humming along—until we hit the dreaded F-word again. But this time, something strange happens: the failed devices share a pattern. Upon investigating, we uncover that all the firewalls in Europe are pointed to the wrong TACACS server. That’s a major design flaw that had slipped under the radar, and it only came to light when RedSeal couldn’t pull the necessary data.

In each of these cases, what appeared initially to be failures, turned out to be incredibly valuable findings. We were able to uncover network design issues that had gone unnoticed until the F-word reared its head.

So, next time you see “Failed,” don’t just assume it’s the end of the world. It might just be the beginning of a crucial network discovery!

Reach out to RedSeal or schedule a demo today today for a personalized walkthrough and discover how RedSeal can revolutionize your approach to cybersecurity.