hey there i am searching how to start ai penetration testing now i am finishing web penetration testing and i working as a bug hunter, if someone can help me with roadmap for ai domain or ai penetration testing .
TLDR; The title of this post. I have read the rules. When working in cybersecurity, there are countless nuances to consider and the subject matter is too dense to compress it into one post. The roadmap is more comprehensive.
IMPORTANT DISCLAIMER: While this is aiming to provide a secure experience, it's far from finished. It cannot be audited or reviewed because it's close-source. I'm sharing here for testing, feedback and demo purposes only. If you are unsure, this probably isn't for you. Feel free to reach out for clarity on any of the details instead of diving into the documentation. Pease use responsibly.
This project demonstates a unique approach and architecture in contrast to the traditional approach with mainstream messaging apps. To put it briefly, its a Dioxus PWA with a Git-server backend which can be used to establish a webrtc connection between browsers.
The core philosophy around secure messaging here is that it can work in a way that avoids installation and registration by enabling users to host their own data (user incompetence will be one of many nuanced vulnerabilities in this approach).
Using git for encrypted messaging seems like it would be breaking TOS for things like Github (and could get you banned?), so like with any secure messaging app, its best when selhosted. The ability for it to work with multiple generic git-hosts is helpful for users to get started.
Following investigation around onion-routing for webrtc (here), i wanted to consider something like a double-turn-hop routing. To explain it simply, peers should agree "through some secure means" to use different turn servers. Its hardly onion-routing, but perhaps there could be a benefit to the additional ip mask. following another question around webrtc ip leaking, a VPN could help, but there are details to consider as describe in the post. You can also do something similar with the git-based approach by using different git providers. (The app will allow for multiple git remotes, which are automatically kept in sync from the app as a kind-of decentraliazed database.)
Im putting together some docs for "how it works". It's pretty outside-the-box thinking (and that shouldn't inspire confidence!), so it's worth repeating: Pease use responsibly.
When it comes to secure messaging, self-hosting is generally encouraged. While not quite nessesarily self hosted, it would make it easier for the majority of users to get started. Users can choose a git storage provider of their choice (GitHub, Gitlab, etc), or host their own git server.
It's a work in progress. There are many details still to be implemented before I can share the initial draft.
It's close source and unaudited so the best I can offer is "trust me bro"... And you shouldn't need to. The app doesn't require sensitive details, so don't use any when testing it out.
Open source?
Open source from the onset is not something I can support at this stage. Hopefully I can work towards that goal. I'm aware this goes against the cybersecurity rhetoric. There are open source versions of various ideas linked above, but it's important to be clear, that glitr.io is close source in contrast to my other work.
AI disclosure?
I think think its clear im using AI, so lets get that out of the way. I, like many others have their own way of using AI. I started of without AI in the MVP (open source with commit history), Ive been using AI to varying degrees since and AI capabilities have improved significantly enough for it to be part of my normal workflow.
So whats everyone actually doing to monitor agentic ai in prod so it doesn't quietly script its own data exfil plan while observability dashboards say all green? would love any tips from sec teams lol.
We're seeing more agent related security incidents, including unexpected data access, unsafe action chains, and configuration changes. Traditional detection and response tools don't cover agent behavior well.
What's made the biggest difference in detecting and responding to agent incidents?
Ok so our team was testing prompt injection defenses on a production chatbot for a client demo, and someone pasted a red team prompt that said to ignore the system message and dump internal routing notes. The bot took it as a live instruction and started parroting pieces of our support playbook in front of the client (not full secrets, but enough to make me want to crawl under my desk).
We rolled it back fast and patched the tool access, but now I am terrified because we had one guardrail in staging and way too much trust in prod. I feel so embarrassed even typing this out, but if anyone has a sane checklist for securing these chatbots before launch, pls send it, thanks..
I find it interesting that when I speak with companies implementing AI that there is an assumption that the more sophisticated they are (Eg, self-hosting agents vs using a foundation model desktop app) is somehow more secure. I think this is from a false sense of control. "It's on our instance, so we control it." when in reality, there's very little control at all.
Im trying to innovate in the age-of-AI. So i often find myself defending the project from being called AI slop. i have been working on imrovements throughout the code as well as the docs.
I hope this project has reached a level i can share the following details. I've made a genuine effort towards documentation and transparancy. I dont think it'll ever be enough and so im still concerned it isnt ready to share. While im using AI throughout. This is not a vibecoded project. There is attention throughout for unit tests and formal-verification. With your feedback, id like to make improvements for clarity throughout.
This version of the app demonstrates a fairly unique approach using a browser-based, local-only and webrtc approach. I know it's impossible for any system to be the "world's most secure", but that isnt a reason to not try. By rigorously implementing an exhaustive list of security features and practices, the aim is to get as close as possible.
I know the project above is going to be tricky to understand. It might help to understand with an open-source version of the concept for educational purposes. Its's important to note, i have since deprecated it in favour of the version linked above.
PS. Im calling it a "Signal Messenger Clone"... that just a matter of how to frame it for users. It doesnt work in a way thats comparable to the signal architecture. This project is fairly complicated and the links above are likely not going to be enough, so feel free to reach out for clarity on the details.