How do you judge an AI agency before you sign?
Choosing an agency is a business decision, so it helps to ask the right questions in the right order.
Most agencies in this field offer some mix of strategy, building and training. The difference is how they start. Some begin with a demo and a price. Others begin with your work. In a typical engagement, you describe a problem, the agency proposes a solution, builds it and hands it over. The risk is that the proposal rests on what you said, not on what actually happens in your team. Good agencies close that gap before they build.
What should they ask you?
Their approach to discovery
Ask how discovery works and who does it. The answer should involve watching people do real work, not only a workshop or a questionnaire.
Their team and who does the work
Ask what you will own at the end: the code, the prompts, the data and the documentation. Ask whether you can run it without the agency, and what it costs to leave. Agencies that answer plainly are usually confident in their work.
Data handling, security and ownership
Ask how your data is handled. Where does it go, who can see it, is it used to train any model and how long is it kept? Ask how permissions are set, what is logged and how the agent is tested before launch. A good agency answers without hesitation and puts the answers in writing. Rules on data differ by country and sector, so have your own adviser check the terms.
Pricing and how scope changes
Ask about cost in plain terms: what is included, what is billed by usage, what changes when scope changes, and what upkeep looks like. We give no figures here because they depend on your work, but a good agency can explain how it will estimate.
Ask about after launch. Who trains your team? Who monitors the solution? How are problems reported and fixed? Software that works on day one still needs care as models, tools and your processes change. An agency that disappears after handover leaves you with a system nobody understands.
Training and life after launch
A short selection process keeps the decision fair. Three areas deserve special attention when you compare.
Do they watch real work?
Discovery depth: does the agency plan to observe real work, or just take your brief? The first leads to a solution that fits. The second leads to a guess.
Who owns what at the end?
Ownership and exit: can you take everything and run it elsewhere? Ask the question early, and notice how the agency reacts. Vague answers now become costly disputes later. Ask for a clear statement of what you own.
How is your data protected?
Training and support: what will your team be able to do on its own at the end? A solution people cannot run or review is a risk, however clever it is.
Keeping control of the project
A good agency is a long-term partner, so choose people you can talk to plainly. Pick the one that listens first, asks about your real work and tells you when a task does not need AI. A partner who says no sometimes is worth more than one who says yes to everything.
AI supports your people and does not replace their judgement. Be wary of any agency that promises otherwise.
Run a small trial or a one-week engagement where you can, so you judge the agency on real work, not on slides. Compare how they behave when things are unclear.
How should you run the selection?
- Write down the problem you want solved
- Ask the same questions of every agency
- Include the people who will use the solution day to day
Which signs show a good fit?
- They explain how they will learn your work before they propose
- They show who does the work and how senior they are
- They put ownership, data handling and training in writing
What should you avoid?
- Promising results before they have seen how you work
- Saying nothing about data, ownership or what happens after launch
- Building a generic product and hoping your team adapts to it later
How our Embedded Week works
Arclight is a frontier AI partner, headquartered in London and operating across the globe. Book a meeting to see how we would approach your company.
Most companies do not need a generic AI product. They need a solution that matches how their teams work: the tools, the hand-offs, the approvals. That is why Arclight begins with the Embedded Week. We sit with your team for a week, watch the tools, the hand-offs, the copy-pasting and the waiting, and map where the time goes.
After that we build a custom solution for your company and train your team to run it. We start small and focused, so you see value early and choose the next step with evidence.
Choosing an AI agency, your questions
The questions leaders ask us most often when choosing an AI agency, with short answers.
What should I ask an AI agency before hiring?
Ask how they will learn your work, what you will own at the end, how they test and monitor, what happens to your data and how your people will be trained. Their answers show how they work.
What are the red flags when choosing an AI agency?
Warning signs include a polished demo with no discovery, promised results before they have seen your work, vague scope, no answer on data or ownership, and a plan that ends at launch with no training.
What does good AI discovery look like?
Good discovery watches real work: the tools, hand-offs and waiting. It ends with a map of where time goes, a ranked list of candidates and a plan you can judge, before any build begins.
Should I pick a large agency or a specialist?
It depends on the work. A large firm brings scale and process. A specialist brings depth and speed on AI itself. Ask who will actually do the work, how senior they are and whether they will still be there after launch.
Why is Arclight's Embedded Week different?
Because we start with your team, not a demo. We sit with you for a week, watch how work really moves, then build a custom solution for your company and train your people to run it. Book a meeting to talk it through.
