If you’ve screened developer resumes in the last year, you know the drill. “ChatGPT,” “Prompt Engineering,” “AI Tools” – sitting in the skills section of practically every candidate. The problem is that typing those words costs nothing. And they tell you nothing about how that person actually works.
Most candidates who claim “AI fluency” are really just users. They open ChatGPT when they remember to, they tab-complete with Copilot when the IDE nudges them. An AI-native developer is a different animal entirely: someone who starts with AI and only drops back to manual work when they have to. Someone for whom losing their AI stack tomorrow would genuinely slow them down.
A standard interview won’t surface that difference. You need different questions. Ones that probe habits and thinking patterns, not tool-name recognition.
This article hands you a ready-to-run protocol: 10 questions, what to listen for in the answers, and the red flags that should make you pause.
Before You Start Asking
One sentence that reframes the whole interview: you’re not looking for someone who knows AI. You’re looking for someone AI has changed.
That’s the whole game. A developer who “knows ChatGPT” can answer every question about tools and say absolutely nothing of substance. An AI-native developer will tell you about a specific problem, a specific workflow, a specific moment the model got something wrong and taught them something in the process.
That’s why this protocol splits into three buckets:
- Workflow: how AI actually shows up in the daily grind
- Problem-solving: how the candidate makes decisions with AI, and without it
- Learning: how they track and adapt to a tool landscape that changes monthly
In every bucket, you’re hunting for specifics, not declarations. “I use AI a lot” is not an answer. “I built myself a script that generates unit tests from a function’s docstring and runs them automatically on every commit” — that’s an answer.
The 10 Questions
Category A: Daily Workflow
Question 1: Walk me through a typical workday. Where exactly does AI show up?
What you’re listening for: A strong answer is specific and anchored in real tasks: “I start my morning going through open PRs — I run those through Claude for a first pass. Then when I pick up a new ticket, I’ll usually sketch the approach out loud to a model before writing a line of code, just to pressure-test it.” The key signal is that AI shows up repeatedly, at multiple points in the day — not just when things get hard.
Red flag: A vague, ungrounded answer — “I use AI for different stuff, mostly for generating code.” No examples from the last few weeks. It sounds like they’re describing how they should use AI, not how they actually do.
Question 2: Show me one tool or workflow you built or configured yourself, with AI’s help, in the last three months.
What you’re listening for: Note the verb — “built” or “configured,” not “used.” An AI-native developer doesn’t just reach for off-the-shelf tools; they assemble things around their own needs. Maybe it’s a custom system prompt wired into their editor, a script that automates a chunk of a pipeline, a small agent built for one recurring task. What matters is that they can explain what it does and why they built it that way.
Red flag: No example at all, or something from “the old days.” They describe tools they use but never anything they built or adapted. That’s a sign AI is still something external to them — a tool to reach for, not part of the toolkit itself.
Question 3: When did AI last let you down mid-task? What happened, and what did you do?
What you’re listening for: Anyone who genuinely leans on AI daily has a dozen of these stories on tap — the model returned code that didn’t run, suggested an approach that wouldn’t scale, hallucinated a library that doesn’t exist. A good answer is concrete and shows the reaction: they caught it, understood why it happened, adjusted. Bonus points if they describe changing their workflow afterward.
Red flag: No example, or a generic non-answer — “yeah, it messes up sometimes, you have to double-check.” That either means they’re not using AI heavily enough to hit its limits, or they’re not catching the mistakes because they’re not actually checking.
Category B: Problem-Solving
Question 4: You get handed a new feature to ship. What’s your first move — and where does AI enter that process?
What you’re listening for: This tests whether AI lives in the thinking, not just the typing. AI-native developers often bring the model in at the planning stage — stress-testing an approach, generating alternatives, surfacing edge cases — before any code exists. It’s also worth noting moments where they deliberately skip AI (see Question 6).
Red flag: AI only shows up during code generation or debugging. That means the candidate treats AI as a typing accelerator, not a thinking partner. It’s not disqualifying on its own, but it suggests they’re not deeply AI-native.
Question 5: How do you decide whether a model’s output is good enough to actually use?
What you’re listening for: This is a test of judgment and maturity. A solid answer names a real validation process: “I read every line and make sure I understand it,” “I run the test suite,” “I check for edge cases the model probably missed,” “I cross-check against the docs for anything API-related.” The candidate should clearly believe the output’s correctness is their responsibility, full stop.
Red flag: “Usually it just works” or “I check that it compiles.” No awareness that code can be syntactically clean and still logically wrong or genuinely unsafe. A candidate who trusts AI output blindly is a liability in production code.
Question 6: What dev tasks do you deliberately do without AI, and why?
What you’re listening for: Counterintuitively, this might be the single best question for spotting real AI-native thinking. Someone who genuinely understands the tool also understands its blind spots. That might be system architecture that hinges on business context the model can’t see, code review where they want to form their own unfiltered opinion, or learning a new technology where auto-generated code would short-circuit their understanding. A deliberate “not here” is a sign of maturity, not resistance.
Red flag: “I use AI for everything” — no reflection on limits. Or the opposite: “I prefer doing most things myself” — which suggests AI isn’t actually load-bearing in their daily workflow at all.
Question 7: What do you do when AI produces working code you don’t actually understand?
What you’re listening for: This is a values question more than a technical one. An AI-native developer should understand every line they ship to production, regardless of who — or what — wrote it. A strong answer sounds like: “I ask the model to walk me through it, break it into pieces, rewrite it myself if I need to really own it.” It’s fundamentally a question about accountability — does this person feel ownership over AI-generated output?
Red flag: “If the tests pass, I leave it” or “I trust the model knows what it’s doing.” That’s a serious risk signal, especially anywhere near production systems, security-sensitive code, or user data.
Category C: Learning and Awareness
Question 8: Which models or AI tools do you actively use, and how do they differ for you in practice?
What you’re listening for: This isn’t about reciting brand names — it’s about whether the candidate deliberately matches tool to task. “I run code review through Claude because it explains its reasoning better. For quick boilerplate I stick with Copilot in the IDE so I don’t break flow. For researching a new library I’d rather use Perplexity because it cites sources” — that kind of answer shows real, opinionated understanding of the differences between tools.
Red flag: They use exactly one tool for everything, with zero reflection on whether that’s actually optimal. Or they can list tool names but can’t articulate how they differ in practice. That’s a user, not a practitioner.
Question 9: What’s changed about how you work over the past year, because of AI?
What you’re listening for: This is a question about trajectory, not current state. A strong answer traces an actual arc: “A year ago I mostly used AI for autocomplete — now I start almost every ticket by talking through the architecture with a model first.” “I changed how I write docs — I draft them in dialogue with AI now.” “I write way more tests, because generating them is fast enough that there’s no excuse not to.” The change should be concrete and, ideally, measurable.
Red flag: No real shift, or something painfully generic like “everything’s just faster now.” If they’re working the exact same way today as a year ago, just with AI humming in the background — they’re an AI-user, not AI-native.
Question 10: If you built your ideal dev environment from scratch today, what would it look like?
What you’re listening for: An open-ended question that gives the candidate room to show how they think about where this is all heading. A strong answer reveals an actual point of view: which models, which integrations, how much automation, where they’d draw the line. An AI-native candidate has thought about this because they’re actively experimenting and tracking what’s shipping in the ecosystem.
Red flag: An answer fixated purely on classic tooling (IDE, CI/CD) with no real mention of AI’s role. Or something so generic it could have been said two years ago — no grounding in what’s actually possible right now.
Optional Add-On: A Live Exercise
Questions only get you so far. If your interview format allows for it, add a 10–15 minute live exercise.
The format is simple: hand the candidate a real, non-trivial technical problem and say, “You’ve got 10–15 minutes. Use whatever tools you want, including AI.”
Don’t grade the solution — grade the process. Watch for:
- Do they reach for AI immediately, or try it solo first? Neither answer is automatically better — but it’s a real signal about habits.
- How do they write the prompt? Do they give the model context, or just lob in a raw question?
- How do they handle a bad output? Do they verify, follow up, or just take the first answer and run?
- Do they actually understand what they got? Ask them to explain any part of the solution on the spot.
What Not to Ask
A few questions that sound reasonable but reveal nothing in practice:
“Do you know ChatGPT / Copilot / Claude?” Everyone asks this. Everyone says yes. Recognizing a brand name says nothing about how someone actually works with it.
“Rate your AI skills from 1 to 10.” Self-assessment is uniquely unreliable here. Candidates who use AI superficially often rate themselves high, precisely because they don’t know what they don’t know. The real practitioners tend to be more skeptical of their own abilities.
“How will AI change the industry in five years?” That’s a question for a columnist, not a candidate. You’ll get a polished answer that says nothing about how this person actually works today.
“Are you worried AI will replace developers?” This surfaces opinions, not competence. Irrelevant to the hiring decision.
The Bottom Line
An AI-native developer isn’t the one who knows every tool on the market. It’s the one whose thinking has actually changed, and who can prove it with specific stories from the last few weeks, not talking points.
Good interview questions don’t test knowledge. They test habits, decisions, and reflection. A candidate who can describe exactly when AI let them down, how they adjusted their workflow afterward, and which tasks they deliberately keep AI out of. That’s someone who genuinely works with AI, not just alongside it.
You can run this protocol on your very next interview.
