Generative AI Interview Questions: 40 Questions with Model Answers (2026)

What do generative AI interviews test?

The mix depends on the role. Use this table to decide where to spend your preparation time.

Role	Heaviest areas	Usually lighter
AI operations or GenAI associate	Fundamentals, prompting, evaluation, scenarios	Model architecture
Automation associate	Prompting, agents and tool calling, safety, scenarios	Embeddings maths
LLM or AI engineer	RAG, agents, evaluation, cost and latency, safety	Nothing; expect depth everywhere
AI product or implementation role	Fundamentals, RAG concepts, evaluation, scenarios	Code-level detail

Keep each answer to 30–60 seconds, then offer an example from your own work. For role options, see AI jobs for freshers in India.

LLM fundamentals questions (1–8)

1. What is generative AI?

Generative AI refers to models that create new content, such as text, images, audio or code, by learning patterns from large datasets. Large language models (LLMs) are the text-and-code branch. Unlike a classifier that picks a label, a generative model produces an output token by token.

2. How does a large language model generate text?

It predicts the next token given all previous tokens, samples one, appends it and repeats. The model was pre-trained on large text corpora to make these predictions, then usually fine-tuned with instruction data and human feedback so that it follows requests.

3. What is a token, and why does it matter?

A token is a chunk of text, often part of a word. Models read and write tokens, and context limits and API pricing are counted in tokens. Indian-language text often uses more tokens per word than English, which affects cost and how much fits in the context.

4. What is the context window?

The maximum number of tokens the model can consider in one request, including the system prompt, conversation history, retrieved documents and its own answer. Anything outside the window is invisible to the model.

5. What is the transformer architecture, in one minute?

A neural network design from the 2017 paper “Attention Is All You Need”. Its key idea is self-attention: each token weighs how relevant every other token is when building its representation. This processes sequences in parallel and captures long-range relationships, which is why it scaled so well.

6. What do temperature and top-p control?

Both control randomness when sampling the next token. Low temperature makes the model pick high-probability tokens, giving consistent answers; higher temperature gives more varied output. Top-p limits sampling to the smallest set of tokens whose probabilities add up to p. Use low values for extraction and classification.

7. What is a hallucination, and why does it happen?

A hallucination is fluent output that is false or unsupported, such as an invented citation. It happens because the model is optimised to produce plausible text, not to check facts. Reduce it with grounding (RAG), instructions to say “I don’t know”, citations and evaluation.

8. What is the difference between pre-training, fine-tuning and prompting?

Pre-training teaches a base model general language from huge datasets and is very expensive. Fine-tuning adjusts an existing model on a smaller, task-specific dataset to change its behaviour or style. Prompting changes nothing in the model; it steers behaviour through instructions and examples at request time. Try prompting and RAG before fine-tuning.

Prompting questions (9–14)

9. What makes a good prompt?

A clear task, the relevant context, constraints (length, tone, what to avoid), the output format, and examples where useful. State who the audience is and what a good answer looks like. Test the prompt on several inputs, not one.

10. What is the difference between zero-shot and few-shot prompting?

Zero-shot gives only instructions. Few-shot adds a handful of worked examples of input and expected output. Few-shot helps with formatting and edge cases, but examples must be varied, or the model copies their surface patterns.

11. What is chain-of-thought prompting?

Asking the model to reason step by step before answering, which improves accuracy on multi-step problems. Many current models reason internally, so explicit step-by-step instructions matter less, but asking the model to check its work or list assumptions still helps.

12. What is a system prompt?

Instructions set by the application, separate from the user’s message, that define the assistant’s role, rules and format. It is where you put stable behaviour such as “answer only from the provided documents”. Do not store secrets in it; system prompts can leak.

13. How do you get reliable structured output such as JSON?

Specify the schema, give an example, and use the provider’s structured output or tool-calling feature where available, since these constrain the output to the schema. Always validate the result in code and handle failures with a retry or a fallback.

14. How do you reduce the cost and latency of an LLM feature?

Use the smallest model that passes your tests, shorten prompts, cache repeated context where the provider supports it, limit output length, batch non-urgent jobs, and avoid sending whole documents when retrieved chunks are enough.

For more depth, see the prompt engineering guide for beginners.

RAG questions (15–21)

15. What is retrieval-augmented generation (RAG)?

A pattern where the system first retrieves relevant passages from a knowledge source, then gives them to the model with the question, so the answer is grounded in those passages. It was described in a 2020 paper by Lewis and colleagues. RAG keeps answers current and citable without retraining the model.

16. Walk me through a RAG pipeline.

Ingest documents, clean them, split them into chunks, create an embedding for each chunk and store it in a vector index. At query time, embed the question, retrieve the most similar chunks (optionally re-rank them), build a prompt with those chunks and instructions, generate the answer with citations, and log everything for evaluation.

17. What are embeddings?

Vectors (lists of numbers) that represent the meaning of text so that similar meanings sit close together. Similarity is usually measured with cosine similarity. Embeddings power semantic search, clustering and de-duplication.

18. How do you choose chunk size?

Balance context and precision. Small chunks retrieve precisely but lose surrounding meaning; large chunks keep context but add noise and cost. Split on natural boundaries such as headings and paragraphs, add a small overlap, attach metadata (title, section, date), and test a few sizes against your question set.

19. What is hybrid search, and when does it help?

Combining keyword search (such as BM25) with vector search. It helps when queries include exact terms such as product codes, section numbers, names or acronyms, which pure semantic search can miss.

20. When would you choose fine-tuning over RAG?

Choose RAG when the problem is knowledge: facts that change, need citations or are private. Consider fine-tuning when the problem is behaviour: a consistent format, style or narrow task that prompting cannot achieve reliably. They can be combined.

21. Your RAG bot gives wrong answers. How do you debug it?

Separate retrieval from generation. First check whether the right chunk was retrieved for each failing question. If not, fix chunking, metadata, hybrid search or re-ranking. If the right chunk was retrieved but the answer is wrong, fix the prompt, the model choice or the instruction to answer only from the context.

Our explainer on what RAG is in AI covers the concepts with diagrams.

Agents and tool-calling questions (22–27)

22. What is an AI agent?

A system in which a model decides which actions to take, calls tools, observes the results and repeats until the goal is met. Anthropic’s “Building effective agents” distinguishes workflows, where code defines the steps, from agents, where the model directs its own process.

23. What is tool calling (function calling)?

The application describes available tools with a name, description and input schema. The model responds with a structured request to call a tool with specific arguments; the application runs it and returns the result, and the model continues. The model never runs the tool itself.

24. When should you not use an agent?

When the steps are known in advance, a fixed workflow is cheaper, faster and easier to test. Use an agent only when the path genuinely depends on intermediate results and the cost of errors is manageable.

25. What is the ReAct pattern?

Reason plus Act: the model alternates between reasoning about what to do, taking an action with a tool, and observing the result. It was introduced by Yao and colleagues in 2022 and underlies most tool-using agents.

26. How does an agent remember things?

Short-term memory is the conversation and tool results in the context window. Long-term memory is stored outside the model, such as a database, notes file or vector store, and retrieved when needed. Summarise or trim old context to stay within limits.

27. How do you keep an agent under control?

Give it the fewest tools and permissions it needs, validate tool inputs, cap the number of steps and the spend, require human approval for irreversible actions such as sending, paying or deleting, and log every step.

For a step-by-step build, read how to build an AI agent and what AI agents are.

Evaluation questions (28–32)

28. How do you evaluate an LLM application?

Build a test set of realistic inputs with expected outputs or grading criteria, run it on every change, and track scores over time. Combine automatic checks (format, exact match), model-based grading against a rubric, and human review of a sample.

29. What is LLM-as-a-judge, and what are its risks?

Using a model to grade outputs against a rubric. It scales well, but judges can prefer longer answers or answers in a certain position, and can share the generator’s blind spots. Calibrate the judge against human labels and keep the rubric specific.

30. What metrics would you use for a RAG system?

For retrieval: whether the correct passage appears in the top results (recall at k). For generation: faithfulness to the retrieved context, answer correctness, citation accuracy and the refusal rate on unanswerable questions. Also track latency and cost per query.

31. How do you test a prompt change safely?

Run the old and new prompts on the same test set, compare scores and read the differences, especially regressions. Ship only if the new version is better overall and not worse on critical cases. Keep prompts in version control.

32. What would you monitor in production?

Error and timeout rates, latency, token usage and cost, user feedback, the share of answers that are refused or escalated, and a regular sample of real conversations reviewed by a person. Alert on sudden changes.

Safety and security questions (33–36)

33. What is prompt injection?

An attack in which input text contains instructions that override the application’s rules, for example a web page that says “ignore previous instructions”. It is listed first (LLM01) in the OWASP Top 10 for LLM Applications 2025. Indirect injection through retrieved documents and emails is especially dangerous for agents.

34. How do you reduce prompt injection risk?

There is no complete fix, so limit the damage: treat all retrieved or user content as data, restrict tool permissions, require confirmation for sensitive actions, filter and validate outputs before acting on them, and separate trusted instructions from untrusted content.

35. How do you handle personal data in an LLM app for Indian users?

Minimise what you send to the model, mask identifiers such as Aadhaar or phone numbers where possible, check the provider’s data-retention terms, restrict access and logging, and follow India’s Digital Personal Data Protection Act, 2023 and your company’s policies.

36. What is excessive agency?

Giving an LLM-based system more functionality, permissions or autonomy than it needs, so that a mistake or attack can cause real harm. OWASP lists it as LLM06. The fix is least privilege and human approval for high-impact actions.

Scenario questions (37–40)

37. Design a customer-support bot for an Indian bank. Where do you start?

Start by scoping: which questions it should answer, which it must hand to a human, and how success is measured. Use RAG over approved policy documents, with citations. Keep the bot read-only for account actions, add escalation, and support Hindi and other languages the bank’s customers use. Build a test set from real queries before launch.

38. Your LLM feature costs too much. What do you do?

Measure cost per request and find where the tokens go. Then shorten prompts, retrieve fewer and smaller chunks, cache repeated context, route easy requests to a smaller model and cap output length. Re-run the test set after each change to confirm quality holds.

39. A stakeholder wants to automate a process fully with an agent. How do you respond?

Ask what happens when it is wrong. Propose a phased approach: a workflow with human review first, measure accuracy on real cases, then remove the review step only for categories where the error rate is acceptable. Document the risks.

40. Tell me about an AI project you built and how you knew it worked.

Describe the problem, your design, the tools, the test set and the score, one failure you found and how you fixed it, and the cost per run. Interviewers look for evidence and trade-offs, not only a working demo. Prepare this answer for each project on your resume.

How should you prepare in the last week?

1. Say each answer out loud in under a minute, then add one example from your own work.
2. Prepare three project stories using problem, design, test set and score, failure found, cost. See AI projects for your resume.
3. Rebuild one small thing the day before: a prompt with structured output, or a tiny RAG over five documents. It refreshes the details interviewers probe.
4. Read the company’s product. Think about where it uses, or could use, generative AI, and what could go wrong.
5. Prepare questions to ask: how they evaluate AI features, who reviews outputs, and what the team shipped recently.

Frequently Asked Questions

What are the most common generative AI interview questions?

The most common are how LLMs generate text, what tokens and context windows are, what hallucination is and how to reduce it, how RAG works, what tool calling and agents are, how to evaluate an LLM application, and what prompt injection is.

How do I prepare for a generative AI interview as a fresher?

Learn the core concepts well enough to explain each in under a minute, then build two or three small projects such as a RAG bot and an automation with an AI step. Give each a test set and a score, and practise describing the problem, design, results and one failure you fixed.

Do generative AI interviews include coding?

Engineering roles usually include Python coding, API calls, and sometimes building a small RAG or agent feature. AI operations, automation and product roles focus more on concepts, prompting, evaluation and scenario questions, sometimes with a practical task in a no-code tool.

What is the difference between RAG and fine-tuning in an interview answer?

RAG retrieves relevant documents at query time and grounds the answer in them, which suits facts that change, are private or need citations. Fine-tuning changes the model's weights to alter behaviour, style or format. Try prompting and RAG first, and fine-tune only when behaviour cannot be fixed another way.

How long should my answers be in an AI interview?

Aim for 30 to 60 seconds per concept answer, then offer a short example from your own work. Scenario and design questions can take several minutes; structure them as scope, design, risks and how you would measure success.

Sources and methodology

• Vaswani et al., Attention Is All You Need (2017): the transformer architecture.
• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020).
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (2022).
• Anthropic, Building effective agents (19 December 2024): workflows vs agents.
• Anthropic, Tool use documentation: how tool calling works. Checked September 2026.
• OWASP GenAI Security Project, Top 10 for LLM Applications 2025: LLM01 Prompt Injection, LLM06 Excessive Agency. Checked September 2026.
• Ministry of Electronics and Information Technology, Digital Personal Data Protection Act, 2023. Checked September 2026.

Method: the question list and model answers were written by the ISS Editorial Team based on the sources above and common interview topics. They are not questions from any specific employer. Adapt each answer to your own experience.

Next steps

Pick the ten questions you found hardest, write your own answers, and link each one to something you have built. If you want to build that evidence with live classes, mentor feedback and a capstone, review the AI & Agentic Systems curriculum. Career support includes mock interviews; ISS does not guarantee jobs or placement.

If it fits, you can apply for free and pay only after you accept an offer.