A software engineering teacher published something quietly devastating: after feeding their entire semester's worth of homework assignments into current AI models, every single one came back passing — most with marks equivalent to a solid B or above. The teacher wasn't running easy tasks. These were carefully designed, rubric-graded assignments meant to build real competency. The bigger signal isn't about students cheating — it's about what "competency-level work" means when AI can produce it on demand. For small teams and agencies, this should feel uncomfortable: most client deliverables are, structurally, exactly what homework assignments are. The pitfall many will fall into is framing this as a detection problem (how do we spot AI use?) rather than a positioning problem (what do we offer that AI can't complete?).
What Is This Actually?
The piece comes from a Substack called "The Last Software Engineer" — a working educator in software and CS who had been designing course assignments that were, by most standards, well-constructed: real problems, clear criteria, rubric-graded output. After noticing student submissions had become suspiciously polished, they ran a controlled experiment: they fed their own assignment prompts, unmodified, into several current AI models. The outputs passed.
What makes this more than an academic footnote is the specificity. This wasn't "AI can explain binary search trees." It was "AI can complete the exact assignment, at the expected level, meeting the stated criteria, within the expected format." The gap between "AI knows about a topic" and "AI can complete a graded task on that topic" has effectively closed.
The technical mechanism is worth understanding. Modern large language models — Claude (Anthropic), the GPT-4o and GPT-5 series (OpenAI), Gemini 2.x (Google) — have crossed a threshold that earlier models hadn't. They can read a rubric, infer the intent behind it, and produce output that satisfies stated criteria without explicit coaching. Earlier AI could generate plausible-sounding prose but consistently stumbled on structured tasks: it would miss rubric criteria, confabulate facts, or produce code that looked fine but didn't run. The 2025–2026 generation doesn't stumble the same way on well-defined problems.
The teacher's response — the genuinely interesting part of the piece — was not to deploy plagiarism detection or issue a ban. They redesigned the course. Assignments shifted toward documented process, live oral explanation, and justification of choices made along the way. In other words: from assessing output to assessing reasoning.
This is a well-established pedagogical principle. Bloom's Taxonomy has always distinguished between "can produce an artifact" (lower cognitive levels) and "can analyze, evaluate, and create with genuine understanding" (higher levels). Most practical coursework lived at the lower levels because they were scalable and easy to grade at volume. AI has now forced — not merely nudged — educators to move up.
The translation for non-educators is direct: any work that can be described as a clear task with an implicit or explicit rubric can likely be completed by a current AI model to an acceptable standard. Acceptable means it meets the criteria. Not brilliant, not contextually wise, not original. But it meets the criteria.
A striking amount of professional services work is about meeting the criteria.
The timeline matters here. By late 2026, widely available consumer AI tools at $20–$30 per month are completing structured knowledge work at that passing grade. This is not a future scenario. It is not an early-adopter edge case.
Why This Matters Right Now
Twelve months ago, this conversation was still somewhat theoretical. AI in late 2024 and early 2025 was genuinely capable but reliably made identifiable mistakes on structured tasks — factual confabulation, missed rubric points, code that ran 90% of the time. You could argue, with reasonable justification, that AI was a production accelerator for the skilled practitioner, not a replacement for skill itself.
That argument now requires significantly more qualification.
Two things shifted simultaneously. First, model reasoning and instruction-following improved enough that well-defined tasks — the kinds with rubrics, explicit criteria, expected formats — are now within reliable reach for current models without hand-holding. Second, the cost and accessibility of these models dropped such that everyone in a professional transaction — clients, contractors, junior hires — has access to essentially the same tools as the practitioner selling the work.
That second part changes the dynamics in a specific way. The educator's crisis wasn't just "AI can do this." It was "students have access to the same AI the instructor uses to evaluate submissions." The professional parallel: clients now have access to the same tools agencies use to produce deliverables. Some clients are already running parallel comparisons. Many more will start doing so without announcing it.
What the story illustrates is that this doesn't arrive all at once. It arrived incrementally for the teacher — increasingly polished work, a growing suspicion, a controlled test. For agencies and freelancers, the pattern is likely to be similar. Not a sudden client revolt, but a gradual shift in what clients consider baseline quality versus premium value.
There's a second-order effect that gets less attention: when AI raises the floor on output quality, it also raises the client's implicit expectations. A solid B is no longer impressive when a client can get a solid B for free. The premium has to live above that line — and it needs to be visible there, not just assumed to exist.
Practical Implications for Small Teams
The educator's pivot — from output assessment to reasoning assessment — maps directly onto a shift small teams and agencies will need to make across their service offerings. Here are four scenarios where this plays out with some concrete specificity.
Scenario 1: The Content Agency
A four-person content agency produces SEO articles, newsletters, and social posts for B2B SaaS clients. Until recently, the value was clear: clients couldn't produce this volume at this quality without a dedicated team. The agency understood the formats, platforms, and brand voice at a level clients couldn't easily replicate.
Today, a client with a clear brief and an AI subscription can produce first drafts that clear the rubric. The agency still adds genuine value — but only above the rubric line. Original research, subject-matter interviews, editorial judgment about which angle resonates with a specific audience at a specific moment: these are things AI cannot produce from a generic prompt. The agencies that have already made the pivot are selling strategy plus production. Those still pricing on word counts are running a model that will face direct pressure.
Scenario 2: The Freelance Developer
A solo developer charges a premium rate to build integrations, automate workflows, and write feature code for early-stage startups. Their clients were often non-technical — which meant, in practice, that clients couldn't easily evaluate code quality themselves. That dynamic has shifted. A non-technical founder can paste code output into an AI model today and receive a quality assessment, an explanation of the architecture, and sometimes a suggested alternative approach.
The developer's value is no longer the code itself. It's the architectural judgment — knowing why this pattern fails under load, knowing where that integration breaks when the third-party API changes in six months, knowing which of the three technically valid approaches actually fits this team's operational reality. Developers who bill for code deliverables will feel margin pressure. Developers who bill for technical judgment and own accountability for production reliability are in a qualitatively different conversation.
Scenario 3: The Internal Training Paradox
A ten-person agency has been growing by hiring junior staff, onboarding them with documented SOPs, and building their skills over 12–18 months into productive mid-level contributors. Those SOPs were, functionally, assignment rubrics. Juniors followed them, produced acceptable output, and gradually internalized the reasoning behind the process through iteration and error.
Here's what we've noticed in how this plays out for growing teams: with AI tools, juniors can now hit the rubric from day one — without internalizing the reasoning. The output looks fine. It looks better than many early-career outputs without AI assistance. But that person can't handle edge cases, client escalations, or any situation the SOP didn't anticipate. Managers who mistake good output for good judgment will be caught off-guard when that situation arrives — and it always arrives.
Scenario 4: The Proposal Problem
A boutique strategy consultancy charges for discovery workshops and written recommendations. A prospective client, before signing, runs the consultant's sample deliverables through an AI model and gets something structurally similar in roughly 90 seconds. They may not know what's missing — the specific industry pattern recognition, the relationship equity that makes recommendations land, accountability for outcomes. But from a rubric perspective, the AI output looks comparable.
This is already happening at agencies across the spectrum. The response isn't to panic. It's to ensure the differentiated value is visible in the deliverable itself, not just present in the mind of the practitioner. A recommendation document that looks identical on the surface but contains genuinely different expertise needs to surface that difference explicitly: the reasoning, the alternatives considered, the specific context that shaped each choice.
How to Respond and Act on This
The educator's move was clean: redesign the assessment so that completing the task isn't sufficient — reasoning must be demonstrated. Here's how small teams translate that principle into operational practice.
Audit your deliverables against a rubric test. List your core deliverable types. For each one, ask honestly: if a client gave a competent AI tool the brief we work from, would the output clear our implicit quality standards? If yes — and for a significant number of common deliverables it will be — that's information you need. The deliverables that pass the rubric test are where your value needs to shift visibly upward.
Shift evaluation from output to process. If you're hiring or vetting contractors, test deliverables alone are no longer reliable signals of genuine skill. A polished writing sample or clean code snippet can be produced with AI assistance regardless of underlying capability. Conduct short structured conversations about the work instead: why this approach, what would you do differently, where did this surprise you. Oral examination isn't a new idea — it's how professional licensing has always worked. It holds up in an AI environment because it requires real-time reasoning that can't be pre-generated.
Document the judgment, not just the output. If your differentiation is expertise and accumulated experience, that expertise needs to leave a visible trace in your deliverables. Not as padding — as explicit reasoning. A content strategy that explains why you chose these topics, what alternatives were rejected and why, what specific audience dynamic shaped the approach: that document is much harder to replicate from a generic prompt because it contains reasoning about a specific client in a specific context. Invisible expertise is a competitive liability right now.
Redesign onboarding around understanding, not output speed. For teams developing junior staff, the core question shifts to: can this person explain why the SOP says what it says? Not just follow it. If the answer is no, the SOP is producing AI-assisted outputs but not developing the practitioner. Some tasks should deliberately preserve the struggle — not as a moral stance against AI, but because the struggle is where genuine understanding develops. Design for it deliberately or it won't happen.
Make accountability explicit in how you position and price. AI tools produce output; they don't own outcomes. A professional who stands behind results, takes responsibility for mistakes, and has genuine skin in the client relationship offers something categorically different from a tool, even a capable one. That distinction needs to be articulated — it won't be assumed.
For tooling: Loom for process walkthroughs that document reasoning alongside output; Notion or a structured docs system for reasoning-first SOPs; Claude or ChatGPT Pro for stress-testing your own deliverables before client delivery, so you know where the AI floor sits before your client discovers it themselves.
AI Work Tools: Where Human Judgment Still Has the Edge
| Tool | Best for | Free plan | Starting price | Key differentiator |
|---|---|---|---|---|
| Claude (Anthropic) | Long-form reasoning, structured tasks, code review | Yes (limited) | ~$20/mo | Strongest at rubric-level instruction following and nuanced multi-step analysis |
| ChatGPT (OpenAI) | Generalist tasks, research, content drafts | Yes (limited) | ~$20/mo | Broadest ecosystem; GPT-5 class handles most structured professional tasks reliably |
| GitHub Copilot | In-IDE code completion and generation | Yes (limited) | ~$10/mo | Deep integration into dev workflow; does not substitute architectural judgment |
| Cursor | AI-first code editor, multi-file context | Yes (limited) | ~$20/mo | More autonomous file editing than Copilot; stronger at larger code scopes |
| Loom | Process walkthroughs, async video documentation | Yes | ~$15/mo | Records human reasoning alongside output — AI cannot replicate this artifact |
| HackerRank | Technical skill assessment | Yes | ~$25/mo per seat | Useful when paired with explanation requirements; rubric-only tests are now weak signals |
The pattern is consistent across the table: tools that capture process and reasoning gain discriminative value in an AI-saturated environment. Tools that assess output alone — standardized code challenges, portfolio submissions, timed writing tests — lose it fast.
What the HN Community Is Saying
The discussion drew 166 comments, a solid signal that this story connected with technical practitioners well beyond the education context. The thread organized around several distinct positions worth mapping.
The largest and most substantive thread came from working engineers and developers noting that this exactly mirrors what they're already seeing in professional code review. Multiple commenters described the pattern of junior developers submitting AI-generated code they clearly didn't write — and couldn't explain when asked. One representative line captured it: the output looks fine until you ask one follow-up question. This precisely matches the teacher's finding. The rubric passes; the understanding isn't there.
A significant skeptical faction pushed back with the calculator analogy. "We've been here with calculators, then Google, then Stack Overflow. The job changes but doesn't disappear." This argument has genuine historical weight, and it'd be intellectually dishonest to dismiss it. The counterpoint that carried more force in the thread is that previous automation waves handled tool tasks while leaving the human judgment layer over the whole process largely intact. A calculator handles arithmetic; it doesn't write the mathematical argument. Current AI is making meaningful inroads on the argument itself. Whether that's a different category of shift or just a larger version of the familiar pattern is a genuinely open question — but "we've adapted before" isn't, on its own, a planning response.
Several educators and instructional designers noted that good pedagogy was already moving toward process-based and oral-examination-style assessment before AI arrived. The ungenerous reading is that this is rationalization; the more useful reading is that it's accurate, and AI simply compressed the timeline for a change that was long overdue.
The most practically useful comments came from agency owners and freelancers quietly acknowledging that clients are starting to ask why this costs X when AI gives them something similar. The consensus response wasn't to defend the old pricing structure but to be explicit about what the premium covers — accountability, context, the judgment to know when the AI answer is wrong, and the relationship that makes the recommendation actually land.
One minority view is worth flagging: some commenters argued that redesigning work to be "AI-resistant" is its own form of artificial difficulty. In professional contexts, AI assistance is often welcome and appropriate. The real question — left somewhat open — is what "supervised AI output" looks like when the supervisor can't distinguish their own understanding from the AI's.
Risks and Things to Watch
The most immediate risk for small teams isn't displacement — it's internal quality erosion that's invisible until it becomes a client problem.
When practitioners routinely use AI to produce first drafts, then lightly edit for tone, they may be clearing the client's rubric on every delivery. The hidden cost is what happens to their capacity for original thinking when they've outsourced the generative struggle for months or years. This is the professional analog of the student who AI-completed every assignment and graduated without mastering the material. The outputs were fine throughout. The person wasn't developing.
The second risk is hiring blind spots. Test deliverables are now unreliable signals of genuine capability for writing, code, analysis, and research summaries. Hiring processes that haven't been updated to verify reasoning — not just output quality — are selecting for strong prompting skill, which may or may not correlate with what the role actually needs when a situation falls outside the standard prompt.
Vendor concentration deserves attention. The AI tools completing this work are concentrated in a small number of providers, and capability shifts between model versions have been significant within single calendar years. Teams that have built client-facing workflows on a specific model's behavioral patterns face disruption when that model updates. Building flexibility across providers — not being deeply locked to one model's quirks — is reasonable operational hygiene at this point.
Finally: expectation inflation. As AI raises the quality floor, client baselines rise with it. What earned approval at a previous quality standard may look unimpressive against current AI-assisted work. Teams that haven't updated their internal benchmarks against current AI capability may be delivering work that looks worse than a free tool — not because the team declined, but because the comparison point moved without them noticing.
Frequently Asked Questions
Does this mean AI will replace small agency teams and freelancers?
Not wholesale, and not soon — but specific business models are under direct pressure, particularly those selling output volume in commoditized formats: SEO content at scale, code by the feature, template-driven reports. What AI cannot replace is the judgment about which output actually matters in a specific context, the relationship that gives a recommendation credibility, and the accountability that comes with professional skin in the outcome. Teams anchored to those things have a defensible position. Teams anchored purely to artifact production need a clear plan for repositioning, and the time to develop that plan is before clients start asking the question directly — not after.
How do I tell if a job candidate used AI to complete their test task?
Rubric-based test tasks are weak signals on their own at this point. The reliable method is a follow-up conversation: ask candidates to walk through their reasoning, explain an alternative approach they considered, or describe what they'd change if a key constraint were different. Genuine understanding handles these questions without hesitation. AI-completed work that wasn't internalized typically doesn't survive even gentle probing. Some teams have moved to short live work sessions — 30-minute Zoom calls where candidates work through a reduced version of a real problem in real time. More expensive to run, considerably more predictive than any asynchronous test.
Should I stop using AI tools for client work?
No — stopping would be a competitive mistake at this stage. The right frame isn't "AI vs. no AI" but "what human judgment layer must exist for this work to justify what we charge?" Use AI as a production accelerator; reserve human judgment for the decisions that actually matter. Draft with AI, decide with expertise, own the outcome professionally. The difference between using AI well and using it poorly is often invisible in the output — which is precisely why teams doing it well need to make their reasoning layer visible to clients, rather than assuming clients will infer it from the deliverable quality.
My junior staff are using AI and their output looks excellent. Is that a problem?
It depends entirely on what you're building toward. If you need high-quality output today and don't need that person to grow into a senior role, AI-assisted output from a junior may be exactly right for the situation. If you're building a team where juniors develop into senior contributors over time, the traditional feedback loop — struggle, error, correction, internalized understanding — needs deliberate preservation. Consider assigning some tasks that explicitly require working through problems without AI assistance. Frame it not as a moral rule but as skill development practice. The goal is people who can supervise AI outputs because they genuinely understand what correct looks like when the AI gets it wrong.
Clients are starting to ask why they should pay my rates when AI gives them something similar. How do I respond?
The honest answer is that AI produces something that clears the rubric, while your team produces something that clears the rubric with the right answer — and more importantly, knows when those two things diverge. Specific industry context, accumulated knowledge of what has failed in similar situations before, the relationship equity that makes a recommendation land rather than get shelved: these are the actual product. The framing that tends to resonate is: AI gives them a draft; they're paying for someone who can tell them whether the draft is correct. That requires a person who can be wrong, knows it, and has enough experience to catch the errors before they cost something real.
Is the calculator analogy actually wrong? Haven't we adapted to every automation wave before?
The analogy is partially right, and it would be intellectually careless to dismiss it. We have adapted to previous automation shifts, and they have consistently created new categories of work alongside eliminating old ones. The historical record is worth taking seriously as a prior. The meaningful distinction in the current moment is that previous waves automated specific procedural steps while leaving human judgment over the overall process largely intact. Current AI is getting better at the judgment layer — not just the steps within it. Whether that constitutes a fundamentally different category of shift or just a larger version of the familiar pattern remains genuinely open. "We've adapted before" is not, on its own, a sufficient planning response. It needs to be accompanied by a specific account of what the adaptation looks like for your particular kind of work.
What types of work are most AI-resistant right now?
Work requiring specific real-world context not in training data: live client situations, proprietary internal data, current organizational dynamics, very recent events. Work where the relationship is structurally the product: strategic advisory, executive coaching, high-stakes negotiation, facilitation. Work where accountability is a core feature: someone must own the outcome, not just produce the artifact. Work at the edges of established knowledge: original research, creative choices where individual taste and cultural context matter, situations where the correct answer doesn't exist in prior examples for AI to pattern-match against. None of these areas are fully immune to AI assistance — AI contributes productively in all of them — but they resist full automation at current capability levels for reasons that are structural, not just temporary limitations.
How should I redesign my service offerings in response to this?
Start by listing what you produce and asking honestly: which of these could a client generate from AI with a well-written brief? Any offering on that list needs to be repositioned around what the AI can't provide in that specific context — client-specific reasoning, accountability, edge-case handling, original thinking, or relationship. Some offerings will change substantially. Others will find the human layer is already visible and defensible without much restructuring. The goal isn't to refuse AI or to pretend the shift isn't happening. The goal is ensuring the parts of your work that genuinely require a human are clearly positioned and priced as such, rather than bundled invisibly into deliverables that look indistinguishable from AI output at the rubric level.
Final Verdict
A teacher redesigning homework assignments is not the real story. The real story is that a clear, documented threshold has been crossed: current AI models can complete adequately defined tasks at an acceptable grade. And "adequately defined tasks at an acceptable grade" is a description that covers enormous amounts of professional services work.
For small teams, freelancers, and agencies, our read is this. The teams under most immediate pressure are those selling output volume in commoditized formats — SEO articles at scale, boilerplate code, template-driven deliverables, research summaries without proprietary insight. Margin compression in these areas is already underway. It will accelerate over the next 12–24 months, not through dramatic client exits but through the gradual shift in what clients consider baseline quality versus a premium worth paying for.
Teams selling judgment, accountability, and work that depends on specific accumulated contextual knowledge are in a different position. Not immune, but not displaced. The value they provide sits above the rubric line — and that's where the defensible premium lives.
The practical action here is not a technology pivot. It's a positioning audit. Go through your service offerings with genuine honesty, identify which ones pass the rubric test when AI gets the brief, and make an explicit decision about each. Some you'll reposition. Some you'll reframe to surface the reasoning layer in a way clients can actually see and value. Some you may exit.
The educator's response was right: don't fight the tool, redesign the assessment. For small teams, the translation is: don't fight the AI, redesign the value proposition.
Teams whose primary offer is "we produce X deliverable at Y quality" should start that conversation now — before clients start it for them. Teams whose work is already anchored in relationship, accountability, and context-specific judgment have more runway. But the same pressure is coming.
The assignments have changed. The work needs to follow.