Tomorrow I’m going to be on a webinar about security, “injecting security into systems engineering.” As Ghandi allegedly said when asked about “western civilization”, “… it would be nice.” I’m going to have to say more than that, and (as you have probably noticed) “saying more” is not something I have a problem with. [eventbrite]
The situation regarding security in systems engineering is dire: people grab technology based on the marketing docs, “it works!” and assume it isn’t crap. Then, they install it with security disabled, or pick easily guessed passwords. As Commentariat(tm) member David Hayward points out: [stderr]
“SolarWinds’ Update Server Could Be Accessed in 2019 Using Password ‘solarwinds123’: Report”
That’s nowhere near as temple-clutching as this one: [reg]
Cisco this week emitted patches for four sets of critical-severity security holes in its products along with other fixes.
The worst of the bugs can be exploited by sending specially crafted IP packets to a vulnerable installation, and overflowing a memory buffer to ultimately execute code as root on the machine, allowing the box to be completely commandeered. Another set of flaws can be abused by sending HTTP requests that trigger arbitrary command execution to again hijack the machine. You should install updates to address these vulnerabilities as soon as possible.
Software-defined networking is Cisco’s big new push for their next generation of expensive shit to sell networking technology. Like with VLANs or access control lists, networkers are going to look at the capabilities that Cisco advertises (“faster! better! easier! scaleable!”) and assume that those capabilities work or are at least present. Remember, when you fly on an airplane with in-flight entertainment, you’re sharing a common network bus with the fly-by-wire controls for the engines – that separation is enforced with VLAN features and access control lists, because running multiple wire runs is extremely expensive on a plane. That separation is assumed to be a functioning feature in the network fabric. It should not make you happy at all to discover that Cisco’s quality assurance is so low that they’ve shipped devices that can be taken over with a single well-crafted packet.
The standard of security and quality in software is so low that Cisco is able to just shrug this off with a patch, because it’s just another software bug. And, by the way, lawyers have carefully crafted shrink-wrap agreements that say, “we will take your money for this product but guarantee in no way that it is suitable for any purpose whatever.” Besides, what can you do? The alternatives are equally garbage. For example: [helpnet]
On Friday evening, SonicWall announced that it “identified a coordinated attack on its internal systems by highly sophisticated threat actors exploiting probable zero-day vulnerabilities on certain SonicWall secure remote access products.”
It’s not a “highly sophisticated threat actor” that broke into your network, SonicWall, it’s that you’re chumps who shipped a product that was vulnerable, and got owned via your own product flaws, just like your customers that relied on you.
Back to the Cisco disaster: there’s a packet-level remote exploit that gives full control of the system. And there’s an application-level remote exploit that gives full control of the system. And there are injection exploits, which are remote exploits in data. This product, security-wise, is inferior to a block of cheese.
Cisco SD-WAN Command Injection Vulnerabilities
These can be exploited by authenticated users to gain root-level privileges on a system running the vulnerable software. This can be achieved via the command-line interface, the tcpdump command, a device template file, and a single-sign-on configuration file. These programming blunders were discovered through a mix of diagnosing customer support tickets and internal security testing at Cisco.
That is not how to discover security flaws. “Our customers stumbled over them” is a really, really, bad excuse. It makes you wonder if they do any product design analysis, review, or testing at all. No, actually, it doesn’t make you wonder.
Cisco DNA Center Command Runner Command Injection Vulnerability
An authenticated remote user can supply a maliciously “crafted input during command execution or via a crafted command runner API call. A successful exploit could allow the attacker to execute arbitrary CLI commands on devices managed by Cisco DNA Center.” It was found during an internal security audit.
That’s security whagarbl for “inputs were not checked” – the ‘command runner API’ just took whatever it was given and handed it off to a shell, probably. That’s a basic code flaw that we started teaching people “don’t do that” around 1986. So Cisco is saying they shipped a product with a complete shit-API. Good to know. Let’s use that in our airplane.
Cisco Smart Software Manager Satellite Web UI Command Injection Vulnerabilities
These bugs can be exploited to run arbitrary commands on a vulnerable installation by sending specially crafted HTTP requests to the web interface. Bugs 1139 and 1141 require authentication and will run the commands as root, and the others require none at all and will run the commands as a high-privilege account.
First let’s emphasize that the commands require authentication in order to run them as administrator, then let’s just slip in the fact that there’s another way around it that doesn’t require authentication. That’s not one, that’s two backdoors, one of which is remote exploitable.
Then, Cisco (“we must look forward, not back!”) forgives itself by pointing out that these vulnerabilities were not detected being exploited in the wild. I.e.: common or garden hackers aren’t using them. Meanwhile, when the classified toolchains of NSA and CIA leak out (as they inevitably do) we discover that they have tons of attacks like these, that they kept closely-held for when they needed to use them against a valuable target. That’s a really important consideration, because it means, “if you’re a valuable target, your opponent can and will dust off stuff that nobody has ever seen before.” Not that that’s hard – apparently Cisco’s customers stumble on this kind of crap all the time.
So, that’s the infrastructure from which companies are expected to build their next-generation high-performance ultra-reliable networks. Remember that the network engineers who build those networks are going to blithely assume that Cisco got their implementation(s) of All The Things right. Which, they rather obviously, haven’t. So there will be security designs and meetings and powerpoints and network diagrams, all predicated on the idea that software-defined networking behaves as the software defines it to behave.
As if any of that matters. I used to do consult calls with clients who were asking me about software-defined networking. Usually, I’d ask them if they were currently using VLANs or copper and/or internal firewalls to build a segmented network. Some of them would say “we tried but it’s too hard” but most of them don’t even segment internally. “Well, in order to build a VLAN/segmented network, you have to architect which systems talk to which and how, and map out connectivity and where critical systems lie. That’s the minimal ground-work for being able to do a software-defined network, as it’s the ‘define’ part of “software-defined“. Cisco and Amazon et al give you tools for doing that, but it’s up to you to define your policy and that depends on you organization’s technology strategy and your segmenting architecture.” You know, the hard part? Many IT managers I’ve talked to seem to think that software-defined networking means they’ll be able to plug everything together and, when it gets congested over there they’ll turn the “bandwidth” knob up to 11. They are horrified when I point out that unless you know which systems are important, you don’t know which systems you can reallocate bandwidth from and to.
There is going to be a great deal of money made, selling crap and then more layers of crap to de-crapify and manage the crap. But in the end it’s going to be crap all the way from the management plane down to the packet level. Every layer of crap is penetrable. Then, along comes some authoritarian jackass who says, “let’s turn cyberspace into a new domain of warfare.”
Jörg says
Recently, I saw this – to put it politely – Jargon Bingo press release from Deutsche Telekom:
https://www.telekom.com/en/media/media-information/archive/deutsche-telekom-s-access-4-0-platform-goes-live-615974
The combination of two
memesconcepts stood out for me: “software-defined networking” and “whitebox hardware”. Surely nothing could go wrong with that!bmiller says
Not to go all CLUB OF ROME on us, but sometimes when a society becomes too complex, too fragile, it will just stop working? Humungus and Mad Max never had to worry about software security. Just knives and grenades and weird slicing boomerangs.
chigau (違う) says
This sounds like is was scripted by Michael Crichton.
cvoinescu says
Jörg @ #1:
I don’t want software-defined telecom infrastructure. Boeing already tried a software-defined airplane, and we all know how well that went. What’s next, software-defined mixed-mode water/sewage networks?
timgueguen says
This is why we probably don’t have to worry about killer AIs. They’ll be designed with a bunch of vulnerable points because it was too much work to design things properly.
Marcus Ranum says
timgueguen@#5:
This is why we probably don’t have to worry about killer AIs. They’ll be designed with a bunch of vulnerable points because it was too much work to design things properly.
And besides the NSA and CIA and PLA and Mossad will all have backdoors into their crypto.
AIs all be going, “HEY WHAT IS THIS STUFF!?”
Actually, there could be a story in there, about an AI who is really depressed because they’re coded in some really amateurish C++
dashdsrdash says
The difference between a junior network engineer, a midcareer network engineer, and a senior engineer:
Junior: Why don’t we use this new feature that our salesperson talked about?
Mid: Because it doesn’t solve any problems better than what we have now.
Senior: Because it won’t work right for several revisions, if it ever does.
Jörg @ #1: whitebox hardware generally means that the vendor of the underlying (chip-level) technology has achieved mass production and is selling reference system plans to everyone who wants them, rather than just dealing with one to five vendors who need to integrate them into their line and make them look like a brand-name product. This can be really good because it no longer carries the price tag of the major vendors, and pretty bad because it can skip QA and technical support.
I would certainly buy whitebox 10g ethernet switches and gigabit access switches at this point; they are relatively mature. That’s what it appears Deutsche Telekom is actually doing here — providing fiber-to-the-house Internet services. The buzzwords are quite thick on the ground, though. Everything about future possibilities is so much hogwash.
Jörg says
Marcus @#6:
A combination of Marvin and Skynet.
cvoinescu says
Marcus, I listened to your webinar. What a ray of sunshine! A man after my own heart.
komarov says
Wouldn’t the AI scenario result in another cold war? The AI is backdoored and the AI in turn backdoors everyone who backdoored it. Immediately you have Strangelove-style failsafe systems running on both sides – scripts to wipe the other side if something happens. The world ends when one side’s failsafe freezes over a bug – I’m betting on the humans – and the other takes advantage. The other world ender is the inverse: The failsafe randomly fires due to bad coding and, since this is all happening in milliseconds, there won’t be any Petrovs around to intervene.
There might be a third scenario, natural causes: the AI drops dead due to bad coding in general or a botched monthly security patch. Shortly afterwards, the dead-bot’s-switch still goes off, of course, rudely interrupting humanity’s celebration of a new era of world peace.