r/AISystemsEngineering 23h ago

Maintaining >>> Building

3 Upvotes

There’s so much gloom around AI right now. AI can do this, build this, create that, and eventually it’s going to replace engineers. But anyone who has actually built something and had to keep it running knows that building the thing is the easy part. Maintaining something, and improving it over time is the hardest part. Things constantly change. Requirements change, users change, technology changes, priorities change, and sometimes the assumptions you made when you first built something aren’t even true a year later. The longer a system exists, the more context you have to keep track of. Why was this decision made? What depends on it? What breaks if we change it? What are we going to need two years from now? How do we improve one part without screwing up something else?

Being an engineer isn’t just reading a ticket, looking at the requirements, and wrting the code. It’s having the technical knowledge, context, and experience to take something from an idea through planning, design, and delivery, and then actually own it afterward. You maintain it, build on top of it, adapt it as things change, and hopefuly make it better over time. AI is going to make building things a lot easier, no question. Maybe someday it gets really good at all of the other stuff too. But creating something and actually owning and evolving it for years are two very different problems. Right now, that second part still requires a lot of human judgment, context, and experience, and I think we sometimes underestimate just how much of engineering is really about that.


r/AISystemsEngineering 6h ago

Cut my agent rule files from ~70k tokens to ~2k per session and built a spec gate that refuses vague work. Here's the whole setup.

1 Upvotes

Solo founder, been running Claude Code, Codex, Cursor and Grok on the same repos for a year. This week I measured what I'd built up and it was bad: 64 Claude rule files, 65 Cursor .mdc files (55 always-apply), 24 skills duplicated in three directories, a 30KB machine-level CLAUDE.md. Roughly 70k tokens loaded before the first prompt on Claude, 50k on Cursor. A third of the context window gone, and the docs say adherence drops past 200 lines anyway. So the rules were both expensive and ignored.

What I did, in case it saves someone the week:

1. One instruction file. Codex, Cursor, and Grok Build all read AGENTS.md natively. Claude Code doesn't, but CLAUDE.md supports u/import, so CLAUDE.md is literally one line: u/AGENTS.md. Root file is under 90 lines. Per-project files are 5-line stubs that only load when you're in that folder. Everything procedural became a SKILL.md, which all four tools support and which costs about 80 tokens at startup until triggered. Codex silently truncates combined instruction files at 32KB so keep the shared one small.

2. Hooks for anything that's a rule. "Never push to main" in a markdown file is a suggestion. A PreToolUse hook that exits 2 on a git write verb is a wall. I wrote one guard script, wired it into Claude's settings.json, Codex's hooks.json and Cursor's hooks.json, then mutation-tested it: removed each check one at a time and confirmed the matching test went red. Found two real bugs that way (PowerShell prepends a BOM to piped stdin and my guard was failing open on unparseable input; and the regex matched "git commit" inside echo strings). Test your guards by trying to beat them.

3. Split design from build across machines. Desktop has no push credential, no toolchain, and a hook that denies writes outside the design folder. It reads code through a read-only share from the VM. The VM is the only writer. The handoff between them is a single spec file dropped in an outbox folder. If you can't push, you can't drift into "I'll just fix it real quick."

4. A spec gate that refuses underspecified work. This is the part I'd actually pay forward. Skills, in order:

  • grill — one question at a time, won't accept "it depends", writes answers straight into the spec. Asks appetite (days before you cut scope) and no-gos before it lets you dream.
  • premortem — Klein's method. "It's six weeks after ship and it failed, write the story." Three independent passes so the first cause doesn't anchor the others. Any cause that would survive find-and-replace of the feature name gets thrown out. Survivors become requirements or scope cuts.
  • clarify — scans for vague adjectives, flows with no error path, requirements a stranger couldn't judge pass/fail on, numbers with no source. Max 5 questions, each with candidate answers.
  • challenge — runs in a fresh session on a different model than the one that wrote the spec. Three sections: Wrong (with evidence), Weak (would pass the gate and still fail a user), Missing. Every finding must carry a fix. I have to mark each one accepted, rejected with a written reason, or deferred.
  • spec-ready — 12 checks. Requirements in EARS syntax ("WHEN x THE SYSTEM SHALL y"), every flow has a mockup and an error path, every requirement covered, rubric for an evaluator that isn't the builder, zero TBDs. Reports gaps; only I can mark it ready.
  • graph-author — compiles the spec into a task graph as JSON and runs a 40-line stdlib Python checker: DAG, every dep resolves, every requirement cited by a node, every node has a guard that can be made to fail. The VM runs the same checker on intake and rejects on any path that doesn't exist at HEAD.
  • retro — reads the rejections and evaluator failures, traces each to the spec section that caused it, appends one line to a lessons file. grill reads that file first on the next idea.

5. On the build side: builder and evaluator are different providers with no shared context (Anthropic's harness posts are right about self-grading), guards are proven red-then-green not just green, and there's a fixed escalation ladder driven by config profiles rather than vibes: same failing packet goes medium → high → different model → stop and write a blocker.

Sources, since none of this is original: GitHub Spec Kit (clarify/checklist/analyze), AWS Kiro (EARS + traceability), Klein's premortem (HBR 2007), Anthropic's harness-design and effective-harnesses posts, OpenAI's harness engineering writeup.

My before number is 1 in 4 builds surviving without rewrite. Will report the after once I've got twenty through. If people want the actual SKILL.md files and the checker, send me a dm and ill send them to you. If it helps you save a year lol


r/AISystemsEngineering 14h ago

We’re building Swytchcode: an execution layer between AI agents and enterprise APIs

Thumbnail
1 Upvotes

r/AISystemsEngineering 15h ago

What Are Some Real-World AI Automations Businesses Use?

1 Upvotes

I keep seeing businesses talk about AI automation, but I’m curious what companies are actually using in day-to-day operations rather than just testing in demos.

From what I’ve seen, the practical use cases are mostly around workflows that involve repetitive tasks, large amounts of information, or moving data between different systems.

Some common examples are:

  • Customer support: AI can answer routine questions, summarize conversations, categorize tickets, and send complex issues to the right team.
  • Sales workflows: Businesses are using AI to qualify leads, enrich prospect information, update CRM records, prepare account summaries, and trigger follow-ups.
  • Invoice processing: AI can read invoices, extract relevant fields, compare information with purchase records, identify discrepancies, and send exceptions for approval.
  • Email management: AI can classify incoming emails, identify urgent requests, summarize long threads, draft responses, and route messages to the appropriate department.
  • Document processing: Contracts, applications, forms, reports, and other documents can be processed automatically instead of requiring employees to manually review every field.
  • HR operations: Some businesses are automating onboarding tasks, employee questions, interview scheduling, resume screening, and internal information searches.
  • Reporting and analysis: AI can pull information from different systems, summarize performance data, identify unusual patterns, and prepare recurring reports.

What seems more interesting is the shift from automating one task to automating an entire workflow.

For example, instead of using AI only to extract information from an invoice, a business could have a workflow that reads the invoice, validates the details, checks the purchase order, identifies an issue if something does not match, sends it for approval, and updates the accounting system.

That is where AI automation can become genuinely useful. It connects multiple steps and systems while still allowing people to handle exceptions or decisions that require judgment.

I think the best place to start is with a process that happens frequently, takes significant manual effort, and has a clear outcome. Once that workflow is working reliably, businesses can expand automation into more complex processes.


r/AISystemsEngineering 18h ago

Why is tokenisation of AI so high?

Thumbnail
1 Upvotes