During our first AI Automation Accelerator, I made a silly mistake.
I try to always keep my educational material as simple as possible. Not requiring additional context wherever possible.
However, early on in the Accelerator jumped straight into discussing the ChatGPT API.
I was assuming that people knew what an API was (they didn’t) and how it was different to “normal” ChatGPT.
Oops. Mea culpa.
If you are currently thinking “what the hell is an API” then you’re in luck! We’re covering that today and why it matters.
For me, the distinction between the ChatGPT app and the API seemed obvious – I'd been building with APIs for years. But the confused messages from several participants quickly made me realise my error. What was second nature to me was completely new territory for many talented entrepreneurs.
That moment was eye-opening. I realised there's a fundamental divide in the AI world – between casual users of web interfaces and those building with APIs. And knowing which to use when is super important for how we build and how we prompt.
This fundamental difference changes everything about how we approach prompting. When we're just chatting with AI through a web interface, we can fudge our way through prompts and iterate until we get decent outputs. But when we're building with the API, we need solid, reliable prompts that work consistently – much closer to the "engineering" part of prompt engineering.
But I'm getting ahead of myself! First, let's look at what the app vs API distinction actually means!
Let's get started:
So far in our series, we've primarily focused on techniques you can use in the standard app versions of AI systems – that's the web applications like ChatGPT, Claude or their mobile app equivalents.
These apps are what we called GUIs (pronounced gooey, which is endlessly funny). GUI stands for Graphical user interfaces. It’s a term from the age of computing when having an graphical interface was actually novel. Before it was all text based (think DOS if you are my age or above!).
Instead of GUI we can just use the catch-all word “app” as it’s close enough. This includes web apps (via a browser like Chrome, going to a website like chatgpt.com) , phone apps like the ChatGPT iPhone app or even desktop apps like the ChatGPT you can install on your computer.
Either way these interfaces are where most people start their AI journey, and for many users, they're perfectly adequate.
Whilst these apps are great starting points, they have significant limitations that become apparent when you start using AI more seriously:
Manual Intervention: Every interaction requires someone to type prompts and copy-paste results, making automation impossible.
Inconsistent Parameters: Settings may change between sessions or updates, affecting output consistency.
Limited Integration: These apps mainly exist as islands, separate from your existing systems and workflows.
Restricted Customisation: You're limited to the features and controls the provider chooses to expose in the interface.
As your AI usage evolves from casual experimentation to core business processes, these limitations start to become real bottlenecks. When we want to start getting sophisticated therefore we need to move away from the apps and towards the API.
Oh great. Another acronym!
API stands for Application Programming Interface. Software engineers aren’t great at naming things so we’ll have to excuse these complex names!
APIs are basically a way for different software systems to communicate with each other. That’s it.
In the context of AI models, an API allows your software to connect directly to the AI service, send prompts and receive answers automatically.
No more fussing about in the app interface writing prompts. No more manual entry. Everything gets sent back and forth behind the scenes without you. VERY different. And essential when you are building automations and software tools that need to work when you aren’t around.
Let’s hammer this home with the key differences:
Using an AI app is like ordering food at a restaurant counter – you're limited to what's on the menu board and how the staff is trained to serve it. Using an API is like having direct access to the kitchen – you can customise ingredients, cooking techniques, and presentation to your exact specifications.
This does not mean that the API is always better than using the app!! There are pros and cons here. Ordering via the menu is much easier and you know what you’ll get. If you head back into the kitchen then yes, absolutely you have more control! BUT you are also more likely to make a huge mess. 🤣
This transition from GUI to API fundamentally changes how we approach prompting. Here's why:
In the app, prompt engineering is often exploratory and iterative:
Basically you can fudge your way through with a combination of zero, one and few shot prompting as we described before, mixed with providing context with file uploads and Projects.
We can get it done. Even if it’s a bit messy! The app versions give us this flexibility.
In API environments, prompt engineering becomes much more rigorous:
This is where prompt engineering truly becomes engineering. You're no longer casually crafting messages – you're designing robust systems that need to work reliably at scale.
Your prompts need to work without you there nudging them along and fixing them on the fly. They need to work whilst you are sleeping. So we need to get more sophisticated. We’ll touch on this shortly, after I explain how exactly we use the API.
Once you decide to move beyond GUI interfaces, there are two main approaches to using APIs:
1. No-Code Tools: Platforms like Zapier, Make, and Bubble let you connect to AI APIs without writing code. You simply:
This is perfect for automating straightforward tasks like generating content based on triggers, analysing incoming data, or connecting AI to other services like email or Slack.
This, by the way, is exactly what we do in the AI Automation Accelerator. We show you how to build your first API-based automation. And more than that how to build something you can actually sell.
2. Custom Development: For more complex needs, you can build custom applications that directly integrate with AI APIs. This requires some programming knowledge but offers maximum flexibility. Again, you'll need an API key to authenticate your requests.
Either way, the fundamental concept is the same: you're using an API key to send and receive messages directly to the model, bypassing the chat interface entirely. This direct connection is what enables automation and consistency. Allowing you to build actual products with AI.
When using the API we have some additional dials we can play with.
One of the most important controls at your disposal is "temperature" – the setting that essentially determines how “creative” or predictable the AI will be.
Temperature controls randomness in the AI's selection process. Without getting into the weeds when generating text, the AI assigns probabilities to possible next words or tokens. Temperature determines how funky it gets with those probabilities:
When using APIs, temperature becomes especially important because you need to deliberately choose the right setting for each use case:
In API settings, you'll often set different temperatures for different endpoint functions within the same application – using low temperatures for data processing and higher temperatures for creative content generation.
How do you know which exact temperature will work best? You can use the above ranges as guidelines but ultimately: testing! When building prompts for repeat use we’ll test results at a range of temperatures and see which works best.
The next big consideration when using the API is dealing with context windows.
Every AI system has limits on how much text it can process at once—its "context window." In API settings, understanding and managing these limits becomes even more critical. Much more so than in the apps where we can just start a new chat!
The context window is the AI's working memory—everything it can "see" at once when generating a response. This includes your prompt, any examples, previous messages, and system instructions.
Context windows are measured in tokens. A token is roughly (don’t come at me!) 3/4 of a word in English, so a 4,000-token context window can handle about 3,000 words of combined prompt and response. Ish. Sorta. More a less.
In API settings, you need to manage context explicitly:
This means that when building with the API we need to actually pay attention to the volume of information we are putting in. How much additional context do we really need to upload? Before with the apps we just chucked everything in and hoped for the best but when building with the API we need to be more strategic.
The good news is that context windows have expanded dramatically:
These numbers are shifting all the time - so whatever I write here will be out of date in a month or two! For the latest information on context window sizes, check comparison sites like Artificial Analysis (artificialanalysis.ai) that track capabilities across different models.
On a positive note though limits are expanding all the time: so the question of context window management may become moot as we move forward. That said efficient token usage will likely remain important for cost, performance and environmental reasons. Just because we could use 1M tokens to help us write a tweet does not mean we should!
All of this may sound intimidating. Sorry! Here’s a nice easy segue into working with this more advanced level of prompt engineering. I strongly recommend spending time in "playground" environments:
These interfaces offer the best of both worlds:
Playgrounds let you:
Think of playgrounds as training wheels for API usage. They let you experiment and refine your approach before committing to full integration. I highly recommend you go and kick the tyres of the API over on a playground to see all of this in action.
Today we've explored the transition from apps to APIs and how this shift fundamentally changes our approach to prompting. We've covered the technical controls like temperature and context management that become crucial when working with APIs.
Tomorrow, we'll examine model selection and optimisation, helping you understand when to use different AI models and how to adapt your approach based on the specific capabilities of each system.