AI Prompt Engineering: Techniques for Better LLM Outputs

· 10 min read

Large language models like GPT-4, Claude, and Gemini are remarkably capable, but the quality of their output depends heavily on the quality of your input. The difference between a vague prompt and a well-crafted one can be the difference between a useless response and a genuinely valuable one. Prompt engineering — the practice of systematically designing inputs to get optimal outputs from LLMs — has emerged as an essential skill for developers, writers, researchers, and anyone who works with AI regularly.

This guide covers the core techniques of prompt engineering, from basic structure to advanced strategies like chain-of-thought reasoning and few-shot learning. Whether you are building an AI-powered application, automating workflows, or just trying to get better answers from ChatGPT, these principles will help you communicate more effectively with any large language model.

Anatomy of an Effective Prompt

A well-structured prompt typically contains several components, not all of which are required for every task, but each of which contributes to clearer, more predictable outputs:

  • Role/Persona — Tell the model who it should be. "You are a senior backend engineer" or "You are a technical writing editor" sets expectations for tone, vocabulary, and depth.
  • Context — Provide the background information the model needs. Include relevant data, constraints, and prior decisions. Models cannot read your mind — if context matters, state it explicitly.
  • Task — Clearly state what you want the model to do. Use action verbs: "Write," "Analyze," "Compare," "Summarize," "Generate," "Refactor."
  • Format — Specify the desired output structure. "Return a JSON object," "Use bullet points," "Write a 3-paragraph summary," "Include code examples in Python."
  • Constraints — Define boundaries. "Do not include external libraries," "Keep the response under 200 words," "Only use information from the provided text."

Here is an example that combines all five components:

You are a senior Python developer specializing in API design.

Context: I'm building a REST API using FastAPI for a task
management application. The API needs to support CRUD operations
for tasks, with each task having a title, description, status
(todo/in_progress/done), priority (low/medium/high), and
timestamps.

Task: Generate the Pydantic models and FastAPI route handlers
for the /tasks endpoint.

Format: Provide the complete code in a single Python file with
clear section comments.

Constraints:
- Use Pydantic v2 syntax
- Include input validation
- Use async/await for all route handlers
- Include proper HTTP status codes and error responses

Zero-Shot vs. Few-Shot Prompting

The terms "zero-shot" and "few-shot" describe how many examples you provide in the prompt to guide the model's behavior:

Zero-Shot

A zero-shot prompt provides no examples — just instructions. The model relies entirely on its training to interpret what you want. This works well for straightforward, well-defined tasks:

Classify the following customer review as "positive",
"negative", or "neutral":

"The shipping was slow, but the product quality exceeded
my expectations. I would buy again."

For simple classification, summarization, and translation tasks, zero-shot prompting often produces excellent results because the model has seen millions of similar examples during training.

Few-Shot

Few-shot prompting includes 2–5 examples of the desired input-output pattern before presenting the actual task. This is dramatically more effective when the task has a specific format, uses custom categories, or requires domain-specific reasoning:

Convert the following natural language descriptions into
SQL queries.

Example 1:
Input: "Find all users who signed up in the last 30 days"
Output: SELECT * FROM users WHERE created_at >= NOW() - INTERVAL '30 days';

Example 2:
Input: "Count orders by status for the current month"
Output: SELECT status, COUNT(*) FROM orders WHERE created_at >= DATE_TRUNC('month', NOW()) GROUP BY status;

Example 3:
Input: "Get the top 5 products by total revenue"
Output: SELECT p.name, SUM(oi.quantity * oi.price) AS revenue FROM products p JOIN order_items oi ON p.id = oi.product_id GROUP BY p.name ORDER BY revenue DESC LIMIT 5;

Now convert this:
Input: "Find all customers who have never placed an order"
Output:

The examples establish a pattern for SQL dialect, formatting conventions, and the level of complexity expected. The model follows this pattern much more reliably than if you simply asked "convert to SQL."

Chain-of-Thought Prompting

Chain-of-thought (CoT) prompting asks the model to show its reasoning process step by step before arriving at a final answer. This technique was introduced by Google researchers in 2022 and has become one of the most effective methods for improving accuracy on complex reasoning tasks.

Without chain-of-thought:

A store sells notebooks for $4 each. If you buy 5 or more,
you get a 20% discount. Tax is 8%. How much do 7 notebooks
cost?

Answer: [model might output an incorrect number]

With chain-of-thought:

A store sells notebooks for $4 each. If you buy 5 or more,
you get a 20% discount. Tax is 8%. How much do 7 notebooks
cost?

Think through this step by step before giving the final answer.

The model then reasons through each step — calculating the subtotal, applying the discount, computing tax — and arrives at the correct answer far more reliably. The key phrase is simply "think through this step by step" or "let's work through this systematically."

Chain-of-thought is particularly valuable for:

  • Math and logic problems
  • Multi-step code debugging
  • Complex analysis with multiple variables
  • Legal or policy interpretation
  • Any task where intermediate reasoning affects the final answer

System Prompts

Most LLM APIs distinguish between system prompts and user prompts. The system prompt sets the model's overall behavior, personality, and constraints for the entire conversation. It runs once at the beginning and persists across all subsequent user messages.

// OpenAI API example
{
  "model": "gpt-4",
  "messages": [
    {
      "role": "system",
      "content": "You are a code review assistant. Analyze code for bugs, security issues, and performance problems. Be concise and specific. Format findings as a numbered list with severity levels (critical/warning/info). Only flag real issues — do not suggest stylistic preferences."
    },
    {
      "role": "user",
      "content": "Review this Python function:\n\ndef get_user(user_id):\n    query = f'SELECT * FROM users WHERE id = {user_id}'\n    return db.execute(query)"
    }
  ]
}

An effective system prompt should:

  • Define the role and expertise level
  • Specify the output format and style
  • Set behavioral boundaries (what the model should and should not do)
  • Establish domain context that applies to all interactions

System prompts in Claude use a similar pattern but through a dedicated system parameter. Gemini uses system_instruction. Despite the different APIs, the principle is the same: separate persistent instructions from per-message content.

Temperature and Sampling Parameters

Beyond the prompt text itself, model parameters significantly affect output quality:

Temperature

Temperature controls the randomness of the model's output, typically ranging from 0 to 2 (with 1 being the default in most APIs):

  • Temperature 0–0.3: Deterministic, focused output. Best for factual questions, code generation, data extraction, and tasks with a single correct answer.
  • Temperature 0.4–0.7: Balanced creativity and coherence. Good for general writing, email drafts, and conversational tasks.
  • Temperature 0.8–1.5: More creative and varied. Useful for brainstorming, creative writing, generating diverse alternatives, and exploring unconventional solutions.

Top-p (Nucleus Sampling)

Top-p limits the model's choices to the smallest set of tokens whose cumulative probability exceeds the threshold p. A top-p of 0.9 means the model considers only the tokens that make up the top 90% of the probability mass. In practice, adjusting either temperature or top-p (but rarely both simultaneously) gives you sufficient control over output diversity.

Max Tokens

The max_tokens parameter sets a hard limit on the length of the generated response. Setting this appropriately prevents the model from rambling and helps control costs. For structured outputs like JSON, set max_tokens generously enough to avoid truncation — a truncated JSON response is worse than a verbose one.

Token Management and Context Windows

Every LLM has a finite context window — the total number of tokens (roughly words) it can process in a single request, including both input and output. Understanding tokens is essential for managing costs and avoiding truncated responses:

  • GPT-4 Turbo: 128K tokens context window
  • Claude 3.5 Sonnet: 200K tokens context window
  • Gemini 1.5 Pro: Up to 2M tokens context window

Practical strategies for token management:

  1. Be concise in prompts: Remove unnecessary filler words, redundant instructions, and irrelevant context. Every token in the prompt is a token that cannot be used for the response.
  2. Summarize long context: If you need to provide a large document, consider summarizing it first or extracting only the relevant sections rather than passing the entire text.
  3. Use structured formats: JSON, YAML, and XML are more token-efficient for structured data than verbose natural language descriptions.
  4. Estimate costs: As a rough guide, 1,000 tokens ≈ 750 English words. A 4,000-token prompt with a 2,000-token response costs based on the total 6,000 tokens processed.

Advanced Techniques

Self-Consistency

Generate multiple responses to the same prompt (using higher temperature), then take the majority answer. This is particularly effective for reasoning tasks where a single chain-of-thought might go wrong, but the most common answer across multiple attempts is usually correct.

Prompt Chaining

Break complex tasks into a sequence of simpler prompts, where the output of each step feeds into the next. For example, to write a technical article: (1) generate an outline, (2) expand each section, (3) review for technical accuracy, (4) edit for clarity. Each step uses a focused prompt with clear instructions.

Retrieval-Augmented Generation (RAG)

Instead of relying solely on the model's training data, retrieve relevant documents from an external knowledge base and include them in the prompt context. This grounds the model's responses in your specific data and dramatically reduces hallucination for domain-specific questions.

Based on the following documentation excerpts, answer
the user's question. Only use information from the provided
excerpts. If the answer is not in the excerpts, say
"I don't have enough information to answer this."

[Retrieved Document 1]
...

[Retrieved Document 2]
...

User question: How do I configure SSL certificates?

Common Mistakes to Avoid

  • Being too vague: "Write something about databases" will produce generic filler. "Write a 500-word comparison of PostgreSQL and MySQL for a SaaS startup handling 10K concurrent users" will produce something useful.
  • Overloading a single prompt: Asking the model to research, analyze, write, format, and proofread in one prompt overwhelms the task. Chain separate prompts for each step.
  • Ignoring output format: If you need JSON, ask for JSON explicitly and provide the schema. If you need markdown, say so. Models will match your format specification closely.
  • Not iterating: Prompt engineering is inherently iterative. Your first prompt is a draft. Examine the output, identify what is wrong, and refine the prompt. Keep a library of prompts that work well for recurring tasks.
  • Assuming the model remembers: In API usage (vs. chat interfaces), each request is independent unless you explicitly include conversation history. Do not assume the model retains context from previous calls.

Build and test prompts interactively — structure system prompts, user messages, and parameters in one place.

Open Prompt Builder →

Best Practices Checklist

Keep this checklist handy when writing prompts for any LLM:

  1. Start with a clear role or persona definition
  2. Provide all necessary context — do not assume prior knowledge
  3. State the task with a specific action verb
  4. Specify the desired output format explicitly
  5. Add constraints to narrow the response space
  6. Use few-shot examples for tasks with specific patterns
  7. Request step-by-step reasoning for complex problems
  8. Set appropriate temperature for the task type
  9. Estimate token usage and set max_tokens accordingly
  10. Iterate on your prompt based on actual outputs

Conclusion

Prompt engineering is not about finding magic phrases or hidden tricks — it is about clear communication. The same principles that make you a good communicator with humans apply to LLMs: be specific, provide context, structure your requests logically, and give examples when the task is ambiguous. The models are powerful enough to handle remarkably complex tasks; the bottleneck is almost always in how clearly we express what we want.

As you build more complex AI-powered workflows, having a structured approach to prompt design becomes essential. Our Prompt Builder tool helps you organize system prompts, user messages, and model parameters in one interface — making it easy to experiment, iterate, and save the prompts that work best for your use cases.