Hiring manager's guide

How to Evaluate AI Fluency When Hiring AI fluency is now a baseline skill.

In 2026, AI fluency is a baseline expectation for most knowledge work hires. The challenge is evaluating it honestly: candidates can fake high-level claims easily. Here are the interview questions and tasks that surface real fluency from someone who has actually built it.

Short answer

Don't ask "do you use AI?" Almost everyone says yes. Instead: ask candidates to describe specific workflows they've built, walk through real prompts they use weekly, and complete a short paid take-home task using Claude or ChatGPT. The gap between surface familiarity and real fluency shows up immediately.

By Bill Colbert · Founder, Treetop Growth Strategy
Published May 2026 · More from the library

The 4 levels of AI fluency

  1. Aware: Has used ChatGPT a few times. Knows what an LLM is. Not fluent.
  2. Casual: Uses Claude or ChatGPT for ad-hoc tasks weekly. Generic prompts. Light leverage.
  3. Operational: Has built personal workflows. Refines prompts. Uses AI daily for 30%+ of work.
  4. Fluent: Has built shared Projects/system prompts. Trains others. Compounds the team's AI capability.

Most knowledge work hires in 2026 should be at level 3+. Leadership roles should be at level 4.

Interview questions that surface real fluency

For any knowledge worker hire

  1. Walk me through the AI workflow you use most often. (Listen for specificity, recent details, refinement over time.)
  2. Show me a prompt you wrote that's been refined over multiple uses. (If they can't, they aren't using AI seriously.)
  3. Tell me about something AI got wrong for you in the last month and what you did. (Tests calibration and verification.)
  4. What's something you tried using AI for and decided not to keep using? (Tests judgment about where AI helps and where it doesn't.)
  5. Which tools do you have personal subscriptions to and which does your current employer provide?

For leadership hires (Director and above)

  1. How have you rolled out AI to a team before? Walk me through the specifics.
  2. Tell me about a workflow where you tried AI rollout and it stalled. What was the issue?
  3. What's your view on AI replacing roles vs. augmenting them, and how does that show up in how you make hiring decisions?
  4. Describe how you'd brief a new direct report on AI usage at our company.
  5. If you started here and wanted to set up your personal AI workflow in week 1, what would you do?

A take-home task that works

Paid 90-minute task: Give the candidate a real (de-identified) work artifact from your team: a customer email thread, a draft proposal, a meeting transcript. Ask them to produce a structured output (synthesis, draft, recommendation) using AI tools of their choice. They share their screen via Loom recording or share the artifacts.

Pay for the time ($100-$200 typical). The signal is enormous: you see how they actually work with AI, not how they describe it.

What fluent vs. non-fluent answers look like

The same interview question produces very different answers depending on where a candidate actually sits on the fluency scale. Here is what hiring managers report hearing at each end of the spectrum.

Low-fluency answers

  • "I use ChatGPT to help me write emails."
  • "I've been experimenting with AI tools lately."
  • "I haven't really had time to get into it yet."
  • "I asked it a question and the answer was pretty good."
  • "I trust my own judgment more than AI output."
  • "I use it for brainstorming sometimes."

High-fluency answers

  • "Here's the system prompt I use for client proposals."
  • "I keep Claude and ChatGPT open for different task types."
  • "This prompt took 6 iterations before it was reliable."
  • "It hallucinated a citation last week. I added a verification step."
  • "I built a shared Project for my team with our brand voice loaded."
  • "I can walk you through my daily workflow if that helps."

High-fluency candidates default to specifics without prompting. Low-fluency candidates use the word "sometimes" heavily and rarely name a specific tool, prompt, or result.

Red flags

Green flags

FAQ

What interview questions reveal real AI fluency?

Ask candidates to walk through a specific workflow they use weekly, show a prompt they have refined over multiple sessions, and describe a time AI gave them wrong output. These three questions separate genuine fluency from surface familiarity faster than any other approach.

How do you test AI fluency in a job interview without being fooled?

The best method is a paid 90-minute take-home task: give candidates a real (de-identified) work artifact and ask them to produce structured output using AI tools of their choice, recorded via Loom. You see exactly how they work, not just what they claim.

Should AI fluency be a required skill for new hires in 2026?

For most knowledge work roles in 2026, yes. The minimum bar is Level 2 (casual weekly use). For leadership roles, Level 3 or 4 is the expectation: built personal workflows, refines prompts, and models AI use for their team.

What is the difference between AI fluency and just using ChatGPT?

Casual ChatGPT use means generic prompts for one-off tasks. Real AI fluency means built and iterated personal workflows, choosing between models based on the task, maintaining a prompt library, and having trained others on what works.

What if a strong senior candidate scores low on AI fluency?

Hire if you have 90 days of onboarding bandwidth and the candidate demonstrates genuine willingness to learn. The real risk is the candidate who is defensive about AI. A coachable level-1 will outperform a resistant level-3 within a year.

Should we evaluate AI fluency differently for executives vs. individual contributors?

Yes. For individual contributors, evaluate daily workflow integration and prompt quality. For executives, prioritize their track record rolling out AI to a team, their philosophy on AI augmentation vs. replacement, and whether they can model fluency for direct reports.

Can you use AI tools to help evaluate a candidate's AI fluency?

Partially. You can use Claude or ChatGPT to review the quality and structure of a candidate's take-home output. But the in-person conversation about how they built that output, what they iterated, and what did not work reveals the most about genuine capability.

Related reading

Want a roadmap built for your business?
The $1,500 AI Audit produces a written, prioritized roadmap in 5 business days.
Book the AI Audit → Take the Gap Assessment
Related

Explore more from Treetop