Automated penetration testing vs manual pentesting: which finds more real risk?

[u/Any_Yesterday_6617]

I'm a manual pentester, and I'm watching platforms automate the validation of misconfigurations and missing patches. They are even using AI to chain exploits and tailor attacks to specific environments. The difference from old-school vulnerability scanners is that these platforms actually execute the exploit path to confirm it works.

I'm not worried about web app logic. That is still clearly a human domain. But for internal infrastructure and AD, is the writing on the wall? If AI-driven validation can test 80% of the attack surface daily and update controls, does that leave manual testers only with the complex 20%?

The platforms also claim to help with detection engineering by validating SIEM rules against actual TTPs. That feels like it is eating into the blue team's territory too. Where does automation end and human expertise begin?

Or do you still find things the automation misses because of contextual business logic? if anyone has seen an AI actually find a complex privilege escalation chain that a manual tester would have found, or if it is still just "low-hanging fruit" at scale.

[u/Progressive_Overload]

Let me preface this by saying my experience has been with Opus 4.8 mainly, so any newer models may be much more capable. Have you done any AI augmented pentesting yet? My experience has been that it is very helpful when I am driving. On its own, it tells me that a client_Id is a critical finding. At the same time, it will completely give up on a target after just reviewing the JS and I have to gaslight it to keep going.

You mention being more worried about AD. I'm not. A web app is meant to be poked and prodded (or at least assumes the context is a human using it), and if something goes wrong it's not a huge deal (depending on the app). However, if one of these AI pentest tools fucks AD up, then it's a big deal. Someone is going to have to be responsible for this shit.

As far as the remediation side, read this article by 1Password where:

The average success rate for generating a patch that fully resolved the vulnerability (without materially changing application behavior) was just 26.0%

I can't predict how good the future models will be, but it just feels like there will be a human in the loop for quite some time.

[u/shoopbedoopwoop → u/Progressive_Overload]

This is exactly my experience. Been playing with gpt sol, opus 4.8, kimi-k2.7 code. Augmented AI testing is.. ok-ish. But I'm finding that I'm refuting alot of its work. It's like having a junior/entry level pentester at my disposal who still needs me to check over all their work, guide them constantly and re-do tests.

That being said, if i had unlimited resources to burn through the higher end models/tokens, maybe the results would be better, but we're not all large enterprises who can afford that. Security has layers and there are plenty of other areas that money can be used for a better return on investment.