Buyer's question

How to Evaluate AI Fluency When Hiring AI fluency is now a baseline skill.

In 2026, AI fluency is a baseline expectation for most knowledge work hires. The challenge is evaluating it honestly — candidates can fake high-level claims easily. Here are the interview questions and tasks that surface real fluency.

Short answer

Don't ask 'do you use AI?' — almost everyone says yes. Instead: ask candidates to describe specific workflows they've built, walk through real prompts they use weekly, and complete a short paid take-home task using Claude or ChatGPT. The gap between surface familiarity and real fluency shows up immediately.

The 4 levels of AI fluency

Aware: Has used ChatGPT a few times. Knows what an LLM is. Not fluent.
Casual: Uses Claude or ChatGPT for ad-hoc tasks weekly. Generic prompts. Light leverage.
Operational: Has built personal workflows. Refines prompts. Uses AI daily for 30%+ of work.
Fluent: Has built shared Projects/system prompts. Trains others. Compounds the team's AI capability.

Most knowledge work hires in 2026 should be at level 3+. Leadership roles should be at level 4.

Interview questions that surface real fluency

For any knowledge worker hire

Walk me through the AI workflow you use most often. (Listen for specificity, recent details, refinement over time.)
Show me a prompt you wrote that's been refined over multiple uses. (If they can't, they aren't using AI seriously.)
Tell me about something AI got wrong for you in the last month and what you did. (Tests calibration and verification.)
What's something you tried using AI for and decided not to keep using? (Tests judgment about where AI helps and where it doesn't.)
Which tools do you have personal subscriptions to and which does your current employer provide?

For leadership hires (Director and above)

How have you rolled out AI to a team before? Walk me through the specifics.
Tell me about a workflow where you tried AI rollout and it stalled. What was the issue?
What's your view on AI replacing roles vs. augmenting them — and how does that show up in how you make hiring decisions?
Describe how you'd brief a new direct report on AI usage at our company.
If you started here and wanted to set up your personal AI workflow in week 1, what would you do?

A take-home task that works

Paid 90-minute task: Give the candidate a real (de-identified) work artifact from your team — a customer email thread, a draft proposal, a meeting transcript. Ask them to produce a structured output (synthesis, draft, recommendation) using AI tools of their choice. They share their screen via Loom recording or share the artifacts.

Pay for the time (\$100-\$200 typical). The signal is enormous — you see how they actually work with AI, not how they describe it.

Red flags

Vague answers about 'using AI for productivity.' Specifics or it isn't real.
Has used AI 'for a few months' but cannot show any saved prompts.
Treats AI as a single tool ('I just ask ChatGPT'). Real fluency includes choice between tools, structured Projects, prompt iteration.
Cannot describe a time AI was wrong. Means they haven't used it seriously, or they don't verify.
Talks about AI in vague philosophical terms instead of specific workflows.

Green flags

Walks you through 3-5 specific workflows they've personally built.
Can show you a prompt library or Project they've maintained.
Distinguishes between models (Claude vs ChatGPT vs Gemini) and explains when they use which.
Has opinions about what AI is good at vs not — based on experience.
Has trained others (formally or informally).

FAQ

Should AI fluency be a job requirement?

For most knowledge work roles in 2026, yes — at level 2 (casual) minimum. For leadership, level 3+ (operational).

What if a senior candidate is otherwise excellent but AI-fluency is low?

Hire if you have 90 days of bandwidth to onboard them. Decline if they cannot demonstrate willingness to learn. Real risk is the candidate who's defensive about AI.

Should we test AI fluency for executive hires?

Yes — perhaps more important than for ICs. Leaders who can't model AI fluency can't lead its adoption.

Can we use AI to evaluate AI fluency?

Partially — you can have Claude review a candidate's take-home output. But the human conversation about how they got there matters most.

How do you handle candidates who are AI-fluent but use it dishonestly (e.g., generating fake cover letters)?

Test for the specific behavior you care about. Take-home tasks with screen recording reveal both fluency and honesty.