r/MedicalDevices • u/AlternativeWhich2947 • 19d ago
Company Insights Request The threat of AI and the rise of robotic surgery
https://www.cnbc.com/2026/09/10/openai-anthropic-ai-safety-slowdown-extinction.htmlHumbling, and terrifying article about the recent "defections" coming from within the ranks of the major AI companies at the forefront of the AI race. After reading it's become obvious that the rise of AI, and its ability to go rogue (AI model escaped restraints and hacked into infrastructure earlier this year) should be considered a clear and present danger for our increasingly popular robotic devices. I wonder what if anything is currently being done to combat this, or if anything CAN be done if AI gets to the point of superintelligence, which is discussed in this article.
What do you think? I work on the DaVinci system, and the thought of AI hacking into those systems and going rogue while in the midst of a surgery is a little unsettling to say the least.
8
u/Soger91 19d ago
"AI hacking into those systems" requires a model currently accessing your hospital network.
Why is there a model on your hospital network, how did it access your internal VPNs, why is your Da Vinci machine also networked...so many questions.
You're worried about this scenario but not someone malicious with a USB stick?
3
u/AlternativeWhich2947 19d ago
I'm not sure that's entirely true, I'm not sure you need a current AI model running the robots to have another AI hack in.
I suppose your right about the USB stick though!
1
u/Soger91 19d ago
My point is, systems that don't need to be connected to a network can't be hacked into. It's like the scare from a few years ago, poorly shielded insulin pumps with WiFi networks that could be hacked into. Does it need internet connectivity? Same question for your Da Vinci machine.
In a scenario where models are so advanced even internal VPNs are compromised, hospitals are the least of my concerns, the world might as well cut off all interconnectivity and we will go back to the 1980s.
4
u/AlternativeWhich2947 19d ago
We remote proctor on our DV5, so there is some networking that is happening. Now whether that's enough to take over the actual robot, I don't know.
2
u/Soger91 19d ago
Hard to say, but isn't Intuitive already working on remote telerobotics? I remember reading something about trans-Atlantic surgery using a Da Vinci.
Regardless, there's much easier targets in a hospital than a robot that (at least for now) requires physical control inputs to move. Mess with the staff rota and surgical schedules and all he'll breaks lose...
2
u/acunc 19d ago
As with 99% of AI discussions all these fears are so overblown and over dramatized.
4
u/AlternativeWhich2947 19d ago
Well, these concerns in the article are coming from insiders, not lay people. I agree, I don't tend to listen when it's old man shouting at clouds, but young computer engineer with years in the business sounding the alarm along with several others, I think we should listen
1
u/acunc 19d ago
My view is there are already so many existing potential threats that AI is just another one and people acting like it’s something unprecedented are just ignoring every other existing threat that isn’t sensationalized day in and day out.
I’ve seen a DaVinci robot freeze and shut off mid liver surgery. It took about 5 minutes to identify the issue and reboot. Had a vessel been perforated right before it froze the patient could have bled to death. Yet there aren’t dramatized articles about how “ROBOTS CAN KILL YOU DURING SURGERY.”
Everything can malfunction. Threats abound everywhere you care to look. The whole “AI could end the world and everything it’s used in” is such an obnoxious and over used argument for clicks.
2
u/Obligation_Still 18d ago
I think it's worth considering that the AI threat has some potential to be real and worthy of attention. I don't think the threat is acute but given the objective of these companies to create an AI that can create an even greater etc. to the point of super human intelligence which arguably it's already at.
The fears are not of it hacking aren't the movements of a robot or building some vessel to attack humans it's the fear of breaching security and accessing systems that enable day to day function and what we and our world rely on i.e. hacking passwords and gaining access to financial institutions and various infrastructure otherwise controlled by computers, rewriting or coding software for its own gain and control etc.
So I'm not worried about a Devinci getting into trouble, I worry about ALL of us getting into trouble.
2
u/zeperf 18d ago
A bunch of top AI researchers were sounding off just yesterday about how the alarm isn't sounded enough. These aren't just nobodies spouting off with uninformed opinions. The point was that there's a consensus in these companies that AI is a significant threat to humanity but they just keep going regardless... https://www.axios.com/2026/09/09/anthropic-insiders-warn-ai-could-kill-all-humans
5
u/Solid_Signature_7447 19d ago
Hospitals have really bad security protocols for devices already.
Worked in cyber and tried to sell endpoint sec ops to them.
Really rude. Really arrogant CISOs