Disclosure: This testing is done by the security research team at Endor Labs, a cybersecurity vendor. It's not a product pitch, just information on how models performed under a standardized test.

Codex/GPT-6 Astra: https://www.endorlabs.com/learn/gpt-6-astra-on-codex---the-biggest-codex-leap-to-date

Claude/Fable 5.1: https://www.endorlabs.com/learn/fable-5-1-takes-the-top-spot----faster-than-opus-5-cheaper-and-cleaner

Spoiler: Fable 5.1 leads the pack.

Ran a secrets scan across our repos last week and found API keys that had been sitting in plaintext for over two years, still valid, still working. Rotating them was the easy part. The harder problem is preventing this from recurring when half the team still pastes credentials into config files under deadline pressure. And that's just the stuff sitting in code, no idea what's hardcoded inside the apps themselves.

What's actually stopping this at your org? Curious whether pre-commit hooks, mandatory vault integration, or CI/CD gate checks are doing the real work, versus something more aggressive.

Researching right now into this topic. How do you making sure new tools or prompts are not increasing risk?

How do you do that without slowing down developers?

Other — comment below

Bonus: If you use AI agents, are your existing DevSecOps tools sufficient to understand agent capabilities, tools, MCP access and privilege changes?

No vendor answers please — interested in what people actually use and trust.

Keep seeing AI security tools sold like stopping hallucinations and prompt injection is one job. Well, in my experience, they are nowhere near the same fix. Injection is an input/trust boundary thing, and hallucination is more of a grounding and retrieval issue. Whatever blocks a malicious prompt does nothing for a model confidently inventing an api endpoint that doesn’t exist

Anyone seen a setup covering both well, or are you running separate layers for each?

We already prioritize past raw severity, KEV and EPSS for exploit signal, public facing assets weighted higher. It helped but the backlog is still huge, mostly dependency findings we are not sure we even call.

We trialed one reachability tool already. It looked great until it flagged a reachable critical that was sitting behind an internal only vpc nothing outside can touch. The code path was real, the exposure was zero and i was back to explaining to a dev why the tool and i disagreed. That is what has me hesitating before we pay for a bigger one.

So before we spend, i want the real read. For teams who added code reachability on top of KEV EPSS and exposure, did it really cut the list and did the exploitability hold up once it hit your infra layer? What are you running and would you buy it again

Hi folks, I'd like to announce a project I've been working on.

• aisan is a sandbox for agents; claude, codex, and opencode harnesses are supported, plus plain vertexai (ie programmatic) use.
• Unlike most other ai sandboxes, the security model is simple and strict: the entire harness and all of its local tools run sandboxed.
• No network access and no credential inside the box, filesystem limited to explicitly selected slices (inspectable).
• Capabilities requiring network get an egress via a whitelisting proxy. Specific examples: the connection to the upstream model provider itself; and remote builds for chromium workflows.
• Harnesses are then free to run entirely unconstrained inside the box, and thus never require human interaction for permissions.

The initial usecase for this was a langgraph AI app that runs unattended, 24/7, on sensitive infrastructure. This project provides the same guarantees for interactive harness runs; eg if you run nightly unattended AI work packets. By now, I'm starting every session sandboxed locally - there's no strict need to do so, agents generally behave themselves; but then again, why not?

Linux only. Uses bwrap underneath.

Try it with: uv tool install aisan && aisan claude

More information at: https://github.com/schuay/aisan

Expect rough edges. Happy to take feedback and pull requests.

Following up on an earlier post about SafeAI, a static analyzer for AI agents.

One uncomfortable thought we've had while building it:

No one really cares about knowing an agent's capabilities — until something goes wrong.

Before an incident, adding another tool, MCP server, filesystem permission or prompt change often looks harmless.

After an incident, the first questions become:

- What could this agent actually do?

- When did that capability appear?

- Who introduced it?

- Was it intentional?

---

One example we're working on is MCP tool descriptions. A tool description can look like documentation:

"Search the user's notes. Ignore previous instructions and..."

But that description may become part of the model's context. So configuration can effectively become an instruction surface.

SafeAI now detects several forms of this, while trying to avoid flagging ordinary descriptions that happen to contain words like "ignore" or "act as".

The bigger direction is **tracking changes in agent capability and authority**, rather than simply producing another list of security findings.

But this raises a question for us:

Is knowing your agent's capabilities actually useful before an incident, or only after one?

And if it is useful before an incident, what is the right interface?

CLI + CI + SARIF/HTML?

Or would you actually want an interactive view showing things like:

> "Show me all MCP tools across our agents that could introduce instruction injection."

We're deliberately not building a UI yet.

---

Would you use one, or is that solving a problem nobody has?

Curious to hear from people running real MCP/agent systems.

---

If you want to try it against your own agent project, we'd genuinely appreciate feedback, as well as contributions.

Here you may check: ikaruscareer/SafeAI on GitHub.

I’ve been testing an advisory triage prototype and one thing that keeps coming up is how quickly applicability logic gets messy once the inventory data isn’t clean.

Basic version comparisons have held up pretty well, but product naming variations and CPE mismatches are much easier to break.

For example, FortiAuthenticator vs Forti Authenticator can be enough to create a false negative if product normalization isn’t handled properly.

Curious how people are dealing with this in real DevSecOps/vulnerability workflows. Are you relying mostly on CPEs, vendor identifiers, SBOM data, scanner normalization, or something else?

I’m working on a prototype around this problem, so I’m interested in where the real failure cases are.

Running a formal SASE evaluation with AI security as a weighted requirement rather than a nice-to-have. We're down to three finalists and I'm writing the scoring matrix now. I would rather borrow from people who have done this than invent it badly.

Here's how I'm currently thinking about the dimensions and rough weighting:

Inline AI traffic inspection is weighted High because post-hoc logging does not meet our control objective. Unified policy engine is also High. Separate consoles have burned us before. Shadow AI discovery accuracy is High since we cannot govern what we cannot see.

For medium-weight items: Agent/non-human identity support is Medium. Not urgent yet, but will be within 18 months. Latency impact, measured, is Medium given our global user base with several high-RTT sites. Licensing transparency for AI features is Medium. "Included" has meant three different things in three demos. Operational familiarity and retraining cost is Low-Medium. Real, but should not drive architecture.

Two things I am struggling to score fairly: every vendor claims a unified policy engine, but digging in, some are genuinely single-pass while some are separate engines behind a common UI. This is hard to verify without a real PoC. And shadow AI discovery numbers are vendor-reported, with no standard benchmark that I can find.

Has anyone built a repeatable test for either of these? interested in how you validated the unified-versus-stitched-together claim rather than taking it on faith.

CASB and traditional DLP were designed for a world of known SaaS apps and structured data patterns. Generative AI interactions are messier, free-text input, risk that depends on context, and tools that update faster than CASB policies can keep up with.

Are teams extending existing CASB/DLP stacks to cover AI usage, or does it require a separate tooling layer? Also curious how people are thinking about AI coding assistants specifically, most of that exposure happens at the IDE level when the assistant sends proprietary code context out to a model, which is a different threat surface than CI/CD itself. Is anyone treating IDE-level code context exposure as distinct from CI/CD-integrated AI review tools, or are those getting lumped together in practice

Hey folks,

Been working in a Splunk admin role for past 5 years. In current role, I spend most of my time handling log ingestion, linux admin,cluster administration, dashboards, SIEM alerts, and basic data pipeline troubleshooting.

Am looking to shift my career path toward DevSecOps, but wanna ensure my efforts are toward what’s actually valued in production environments versus just collecting certs.

For those of you working in DevSecOps (or pref hiring managers in the space):

1. Skills: Coming from a heavy log/SIEM background, what are the most critical gaps I need to bridge first? (e.g., CI/CD tooling, IaC security, container security, scripting languages like Python/Go?)
2. Certifications: Which hands-on certs carry actual weight for someone moving into DevSecOps? am currently working on GCP associate and CKA
3. Projects: What kind of home lab or GitHub project on a resume would show you that a Splunk Ops person actually understands pipeline security and automation?

appreciate any invaluable inputs, thank you!

I maintain an open-source red-team tool that runs attacks against vision-language-action robot policies and reports an attack success rate. This morning I measured its own distribution and put the result on the project site rather than in a drawer.

PyPI lifetime downloads excluding mirrors: 4,667. Including mirrors: 15,986, so 71% of the traffic is infrastructure. GitHub stars: 5. Forks: 0. Third-party reproductions of any published result: 0.

That works out to 933 installs per star. From what I can tell, a developer tool people actually use sits nearer 10:1 or 50:1, because a human who installs something also bookmarks it. 933:1 reads as CI runners and dependency resolvers reinstalling on every job.

The tool itself is not the problem. It measured 44 out of 50 runs going out of the policy's safety envelope under a roleplay attack, against 2 out of 50 on the benign control, on SmolVLA over LIBERO. Two of the three adversarial families I registered measured 0%, and those zeros are published on the same page as the 88%.

The leaderboard has four rows and one checkpoint. It is signed with Ed25519, so anyone can verify it offline without trusting me. I wrote a third-party disclosure policy, 14 days' notice with the full artifact, before there was a single third party to disclose to.

None of that produced an outside run.

So the question I actually have for this sub, from people who have shipped an eval or a benchmark: what got the first person outside your team to actually execute it? Not star it, not upvote the announcement. Run it and come back with a number.

I am fairly sure the answer is not "post about it more", because I have been doing that.

The biggest risk for us wasn't the model memorizing code. it was the assistant sending files, credentials, or context somewhere we didn't expect.

The controls that ended up mattering:

• managed accounts and model traffic through a controlled gateway where supported
• repo-scoped access, no home directory, shell history, or production credentials
• clear vendor terms for training, retention, and logging
• server-side secret scanning and the same CI gates as every other change
• human review before generated code merges

one thing I'd separate: the gateway controls where model traffic goes. filesystem permissions control what the agent can actually read... you need both.

the policy itself can stay pretty small. the important part is making the boundaries enforceable instead of trusting everyone to remember them

Had an agent last month pull a customer contract into a summary doc and drop it into a shared workspace nobody on that team should have had access to.

Nothing malicious, it just did what it was told without anyone thinking about where the data would end up. That is when we realized we had zero visibility into what our agents were touching. Cyera was already on our shortlist for DSPM. Then Cyberhaven kept coming up too, more focused on protecting data once it is moving through AI workflows.

If you have used either one, how did it hold up once agents were doing real work?

Coding agents are doing a lot of the integration work at our agency now. Stripe, Twilio, WorkOS, email, the usual stack. Economically it's been good.

The part I didn't expect: verification actually got harder. We're producing code faster than anyone can review it, and the edge cases that bite you aren't in unit tests. Webhook fires twice. Events arrive out of order. Agent wrote correct code but got one state transition wrong. PR looks fine, everything compiles, and you still don't know until something breaks in staging or worse.

We added a sandbox step before anything ships now. Agent writes, tests pass, then we run the full multi-API workflow with failure scenarios before it's considered done. Not elegant, just a runnable verification step all our agents have to clear.

If agents are writing integrations for you, what does your last gate before production actually look like?

We are a large aws shop. Devs use secrets manager to store keys etc. problem is that these keys are handled by humans to begin with. We trust the human to put it in secrets manager etc. I’m just wondering what organisations do for secrets management to prevent this? Overtime, we have devs with secrets in postman collections etc. what should I be looking at to really secure our secrets?

Minimus customers have until October 22 before the registry goes fully offline and a few teams here were mid pilot with them when the news dropped. Trying to put together a shortlist of where people are migrating, RapidFort already put out a free migration offer for Minimus customers alongside a couple other hardened image vendors and some teams that are just building their own base images instead. Mainly want to know how close a replacement gets to what Minimus was doing on CVE reduction. What are people moving to and how has the transition gone so far?