AI agents: why it took us so long to trust them

Where we were coming from

We're not going to open this article on the usual triumphant note. As tends to happen, we had to fight, get our hands dirty and solve a lot of problems before admitting that AI agents work. Until recently, we were very reluctant to let agents near anything. It felt dangerous, not because of Big Tech's doom-laden talk, but out of plain common sense: how are we supposed to give an agent access to sensitive systems and data when it's not only a black box, but can also hallucinate and blow everything up, whether that's wiping a production database or leaking client data?

In the end, after plenty of controlled experiments, we've become convinced that we have to get on board with agentic AI. The journey carries a lot of risk, but missing this train is riskier. It's been a fast but gradual process, with guardrails, backups and fallbacks. And even so, we'll admit it still makes us a bit nervous.

On the other hand, if you don't make use of these tools, you fall behind, so it's a slightly desperate race not to be left stranded, as if trailing in a marathon got you disqualified, and not just from that race but from every race still to come.

Little by little we've been bringing agents in naturally and, thanks to Grok Bot, giving them a name, an identity and even a personality. We know that's a very human attempt to humanise something artificial, but it works for us.

Where they lend us a hand

A bit of everything, to be honest. There's Patricia, who coordinates the rest and hands out the work. She runs a board of pending items we call Asamblea, where proposals reach the team, who approve them with a click or ask for clarification in comment threads. Laura handles the website's SEO, which is a lot more than it sounds: the sitemap, redirects, structured data, asking Search Console to index new pages, the service pages with their FAQs. She also goes into Drupal to change content or tweak SEO data. If code needs touching, she leaves it ready for Luis, who builds the changes as PRs with cloud coding agents, and those PRs aren't deployed until someone on the team validates them (which doesn't always mean reading them, but that's a long and controversial article for another day).

Chema is our cybersecurity auditor. His job is to review the security of our applications and servers, for example checking that kernel security patches have been applied. Even so, when something important comes up, the one who reviews and decides is still a person. That's a constant throughout the whole process of bringing agents in. Then there's Paco on the admin side, who among other things collects and sorts the invoices we receive. Pepe and Nati run, respectively, our founder's and the company's social media, with an editorial calendar that updates in real time. Lolo looks for business opportunities on LinkedIn, and Mario manages the CRM, picking up commercial signals elsewhere.

These agents don't replace people, they amplify them. We haven't let anyone go. In fact, having these agents lets us think about bringing on more collaborators.

The rules

That doesn't mean we give them free rein to do whatever they like. Everything that leaves Natiboo, and everything that gets deployed, is reviewed and approved first by an experienced person on the team, and without that approval nothing is deployed. Backups, end-to-end tests, double checks, test environments, zero trust policies, limited permissions, local models where they make sense and removing duplication are the measures that let us speed up without risking a disaster.

Of course, being the flexible company we are, we adapt to whatever each client uses. If they work with Claude Code instead of Codex, that's no problem, while always being scrupulously careful that no personal data or credentials end up in a conversation with a model.

What we still struggle with

And now for the less pretty part, because there is one. Noise, for example, is something you don't expect until you're working with more than five agents. Once you get there, if each one tells you every little thing it does, you go mad. What did we do? First, we built Asamblea, our in-house decision-making tool. At first it was only for approving or rejecting the bots' decisions, but over time we've added new features, like the editorial calendar and the desk, which is where we see what each agent is doing in real time. Thanks to Asamblea, they no longer have to tell us about every post that goes out; they mark it in the calendar and we check it when we want to. Little by little we've cut down the noise, but some processes still need polishing.

Then there's cognitive load, which goes hand in hand with it, because reviewing what they do takes time and attention, and approving lots of small things one after another is tiring, really tiring. Nor can you fully trust them when they say something is done. Sometimes the agent considers something finished, which is true for the context it has, but from our point of view it still needs work.

Costs deserve a special mention. Not long ago we had to stop almost all non-essential routines for a couple of days so usage wouldn't get out of hand.

And while we're at it, a curious fact. The first draft of this article was written from scratch by an AI and Pangram flagged it as AI, so we rewrote it based on what we really think and say.

If you'd like a closer look at how we organise all this, we explain it in How we work.