Free Starter kit · Indian School of Skills
AI Projects Starter Kit
Many AI portfolios are a chatbot wrapped around an API. What stands out is evidence that you understand the problem, the failure modes and how you measured quality. These six briefs go from beginner to advanced; each one ends with an evaluation step, because that is what makes it credible.
1. Six project briefs
Project 1 (beginner) — Prompt workflow with a test set
Build: a prompt that turns messy inputs (e.g. customer reviews) into a structured summary: sentiment, main issue, suggested reply.
Evaluate: create 30–50 test inputs with expected outputs; compare 3 prompt versions and report accuracy and failure types.
Project 2 (beginner) — No-code / low-code automation
Build: an automation in n8n, Zapier or Make that takes form submissions or emails, classifies them with an LLM and routes them (sheet, Slack, CRM).
Evaluate: measure classification accuracy on real-looking samples; add a human-review step for low-confidence cases and explain why.
Project 3 (intermediate) — Structured data extraction
Build: extract fields from documents such as invoices, job descriptions or resumes into JSON with a fixed schema, validating the output.
Evaluate: field-level precision/recall on a labelled sample; show how schema validation and retries reduce errors.
Project 4 (intermediate) — Retrieval-augmented generation (RAG) over documents
Build: a question-answering assistant over a document set you are allowed to use (public policy PDFs, product docs, your college handbook). Chunk, embed, retrieve and answer with citations.
Evaluate: a question set with known answers; measure retrieval hit rate and answer correctness separately; test what happens when the answer isn't in the documents.
Project 5 (advanced) — Tool-using agent
Build: an agent that uses 2–4 tools (search, calculator, a database query, a calendar or ticketing API) to complete a multi-step task such as researching and drafting a comparison.
Evaluate: task success rate on 15–20 scenarios, number of steps, cost and latency per task, and a list of failure cases (wrong tool, loops, hallucinated arguments) with fixes.
Project 6 (advanced) — Evaluation harness and guardrails
Build: a reusable test harness for one of your earlier projects: test cases, automatic checks (exact match, schema, LLM-as-judge with a rubric) and a report you can re-run after every change.
Evaluate: show a before/after comparison when you change the model, prompt or retrieval settings; add checks for prompt injection and unsafe outputs.
2. How to document an AI project (README checklist)
- The problem and who it's for, in two sentences.
- A short demo (GIF or 60-second video) and how to run it locally.
- Architecture diagram: inputs, model calls, tools, storage.
- Model and settings used, and why (cost, speed, quality trade-off).
- Evaluation method, test-set size and results table.
- Known failure modes and limitations, stated honestly.
- Cost and latency per request (approximate is fine; say how you measured).
- Data sources and permissions; how secrets and personal data are handled.
- What you would do next.
3. Concepts to be able to explain
- Tokens, context windows, and temperature.
- Prompting patterns: clear instructions, examples (few-shot), structured output, step-by-step reasoning.
- Embeddings and vector search; chunking strategies.
- RAG vs fine-tuning: when each is appropriate.
- Function / tool calling and how agents plan multi-step work.
- Hallucination, prompt injection and data leakage risks, and mitigations.
- Evaluation: offline test sets, human review, LLM-as-judge and its biases.
4. Interview questions on LLMs, RAG and agents
- What is the difference between generative AI and agentic AI?
- Explain how RAG works, step by step.
- How do you choose chunk size and overlap for a RAG system?
- Your RAG assistant gives confident wrong answers. How do you debug it?
- When would you fine-tune a model instead of using RAG or better prompts?
- How does tool / function calling work?
- How do you stop an agent from looping or taking unsafe actions?
- How would you evaluate an LLM feature before launch?
- What is prompt injection, and how do you defend against it?
- How do you reduce cost and latency in an LLM application?
- What are the trade-offs between an API model and an open-weight model you host?
- Walk me through a project you built. What failed and how did you fix it?
Want structured feedback on this?
The ISS AI & Agentic Systems program is a live online cohort where mentors critique the projects you build. Talk to an advisor to see whether it fits your goals, or explore the curriculum first.