💬 An Intro To Prompt Engineering

Introduction

What is a prompt?

A prompt is the text you provide to a large language model (LLM) to elicit a helpful response. Prompts are generally structured as questions or instructions to guide the LLM to generate useful information or complete tasks.

When you send a prompt, the LLM analyses the text to understand what you are asking for or what problem you want solved. It then tries to provide the most appropriate response based on what it has learned from its training data.

Some examples of prompts are:

"What is the capital of France?"

"Please summarise this article in two sentences."

"Can you translate this paragraph from Spanish to English?"

The text that the LLM responds with is called the output. For example:

Understanding exactly how prompts work allows you to craft them effectively to get the outputs you desire from an LLM.

How LLMs work

LLMs are large neural networks trained on massive datasets of text to predict sequences of words. This means LLMs generate their responses sequentially, choosing each token (word or punctuation) one at a time based on the words that come before it.

LLMs cannot see your entire prompt all at once - they are limited by their context window size. The maximum context window varies by model, but is often tens or hundreds of thousands of words. Any text beyond the context size will not be seen by the LLM when generating a response.

👉 OpenAI's GPT-3.5-turbo-16k has a context window of ~ 12,000 words / 16,000 tokens
👉 Anthropic's Claude currently has context window of ~75,000 words / ~100,000 tokens

LLMs also cannot go back and edit their responses after generating them, unless you explicitly prompt them to do so. Their predictions are probabilistic and chain off previous tokens selected.

Knowing these key aspects of how LLMs work enables you to structure prompts in a way that takes advantage of their capabilities. Delineating clear sections and keeping text within context limitations will lead to better results.

Why good prompt design matters

Carefully structuring prompts is crucial to guiding LLMs to provide useful and accurate responses. Without proper prompting techniques, their outputs may be generic, inconsistent, or even nonsensical.

Some key reasons good prompt design makes a major difference include:

• LLMs have no other context beyond what you provide in the prompt. Any key background information needs to be stated explicitly.

• LLMs cannot remember previous conversations or look up external information online unless you include it directly in the prompt.

• LLMs try very literally to do what you have instructed them to do based on the prompt. Clear, detailed prompting leads to more accurate execution of tasks.

• LLMs generate text sequentially. Structuring prompts methodically reinforces logical, reasoned thinking.

• LLMs may "hallucinate" or make up information if asked open-ended questions they do not know the answer to. Prompts should be designed defensively to avoid this!

Putting effort into learning prompt design principles and techniques will enable you to clearly explain the purpose of your prompts. This allows the LLM to have the context necessary to provide high-quality responses tailored to your specific needs.

Experiment with your prompts

Large language models can benefit from an iterative process of experimentation and refinement when tackling a new task.

1. Formulate a prompt that provides examples and rules to guide the model's behavior. This is like an experimental protocol.

2. Provide input to the model and observe its output. This is like running trials.

3. Analyse the results to see where the model is succeeding versus failing. Look for patterns in the failures.

4. Adjust the prompt based on the analysis. For example:

   • Make the rules more explicit

   • Add new examples and desired outputs

   • Provide more training data of relevant types

5. Repeat steps 2-4, iteratively refining the prompt.

Additionally, the model can be useful for checking its own work by providing its previous outputs back as input. For example:

• Ask the model to evaluate, fix, or choose between multiple responses it previously generated.

• Prompt the model to double-check its work and find any remaining errors.

When asking the model to find or evaluate something, it helps to provide an "out" so it avoids hallucinating nonexistent issues. Tell the model what to do if it doesn't find anything.

Prompt Chaining

A powerful technique to handle tasks with identifiable and well-structured subcomponents is to sequentially process a model's response through multiple stages. This approach is referred to as prompt chaining.

Prompt chaining enables the accomplishment of multi-step tasks by breaking them down into simpler, more manageable sub-tasks, rather than delivering a complex and comprehensive instruction. This method can often be more effective and efficient than bundling all sub-tasks into a single directive.

Converting a complex directive into a sequence of prompts has several benefits:

• It simplifies the creation of instructions.

• It helps in isolating and troubleshooting specific segments that the model may struggle with.

• It allows for gradual evaluation of the model's output rather than just a final assessment.

Here are a few practical applications of prompt chaining.

Combining Text Analysis with Specific Answers

Consider needing an analysis of a historical document to respond to a particular question. The process can be divided into two distinct prompts:

Prompt 1: Analysing text for specific information

Prompt 2: Formulating a response using the extracted information

Implementing a two-stage review for error detection

A two-step process can be used to identify and confirm errors, such as grammatical mistakes in an article.

Prompt 1: Initial error detection

Prompt 2: Confirmation of rrrors

Coordinating multiple related tasks

In scenarios where related tasks need to be executed simultaneously, such as explaining a scientific concept to readers at various educational levels, prompt chaining can be used in a coordinated manner.

Prompt 1: Creating a targeted outline for different readers

Prompt 2: Transforming the outline into detailed explanations

By using prompt chaining, each of these examples breaks down complex assignments into more focused and manageable steps, allowing for clearer instruction, targeted troubleshooting, and more controlled validation of the model's responses.

Using Parameters

You'll typically use temperature as the most commonly used parameters to alter the model output.

temperature - A measure of how often the model outputs a less likely token (on a measure of 0 to 1).

The higher the temperature, the more random (and usually creative) the output. This, however, is not the same as “truthfulness”. For most factual use cases such as data extraction, and truthful Q&A, the temperature of 0 is best.

A couple of other parameters you'll come across:

maximum number of tokens (maximum length) - Does not control the length of the output, but a hard cutoff limit for token generation. Ideally you won’t hit this limit often, as your model will stop either when it thinks it’s finished, or when it hits a stop sequence you defined.

stop (stop sequences) - A set of characters (i.e. tokens) that, when generated, will cause the text generation to stop.

More resources

Anthropic's guide to prompt design

OpenAI's best practices for prompt engineering

Join the discussion

For help, tips and advice on building AI-powered apps, tools and automations, join our community of no-code makers and power up your progress.