How to Autonomate your Agency

How to Autonomate your Agency - Insight Platforms

If you don't fancy reading, you can watch instead ... it only takes 2 minutes. Sorry about all the stock clips. Can't afford the good stuff yet.

At the recent IIEX Europe conference, Elaine Rodrigo - Danone's Chief Strategy & Insights Officer - used Humanise Data as the theme for her presentation. It showed how domain experts in her team generate insight by applying judgement and experience to complex data analytics and machine learning tools.

Using different labels, this was a theme that recurred in other presentations. There's a sense that something has been lost as we've followed the tech mega-platforms in data-fying humans for marketing. More insight teams are acknowledging, like Danone, that we need to counter-balance this with humanised data.

But what does this really mean in practice? That last paragraph is just garbage if you read it in isolation.

Of course we need to combine what's human as we roll out tech platforms and data tools. But this also applies to the process of research delivery - not just the analytics.

So this post is a re-hash of an old article that riffs on this theme: some practical suggestions for what it really means to merge people and technology in research agencies.

The future of agencies will be autonomated

No, that's not a typo.

Autonomation - or Jidoka - is a feature of Lean Manufacturing.

It means ‘intelligent automation’, or ‘automation with a human touch’. It is one of many innovations pioneered by Toyota to avoid or remedy defects (charmingly known as Poka Yoke) - and increase productivity and quality.

It’s a concept that agencies should warm to as they grapple with the challenges of automation.

So what does it mean in practice? Let’s start by looking at what it is not: a typical project workflow for a research agency.

[Yeah, I know this isn’t everyone … but if you’ve worked in an agency, you’ll definitely recognise it]

Today’s typical agency process

Clients pass their time and budget pressures to their agencies. Over the years, most agencies - even quite small ones - have tried to trim costs and shorten deadlines with a combination of:

  1. Specialisation by function: people can’t be great at everything.
  2. Core competence focus: outsource the stuff you don’t want to do.

On the surface, this sounds like it should make sense.

But what it means is that quite simple projects often have many different stages and teams. Take a common one, like an online quantitative study. A simple workflow - in big and small agencies - might look like this:

  1. Client briefs account manager.
  2. Account manager designs study.
  3. Client signs off study design.
  4. Account manager briefs project manager.
  5. Project manager sends the survey document to programming team / vendor.
  6. Project manager briefs online panel team / provider.
  7. Project manager briefs data processing team / external partner.
  8. Study launches.
  9. Panel team / provider sends results to data processing team.
  10. Data processing vendor sends formatted data and tables to project manager.
  11. Account manager begins working with results.

This sort of process has very limited application. It can work for large batch production, say for on-going panels or global trackers. But that’s kind of it. It’s pretty terrible for custom, fast turnaround projects.

This is what often happens next:

  1. Account manager discovers errors in the data.
  2. It’s Friday afternoon.
  3. Monday morning. Go back to step 4. Diagnose the problem.
  4. Researcher updates the survey design error in the document.
  5. Researcher sends it to the project manager.
  6. Project manager sends it for re-programming.
  7. Project manager sends it for re-fielding and re-running data.
  8. The client calls for a progress update. Cross words are exchanged.
  9. etc

Dysfunctional as it sounds, this is common. And that’s before we even get to the late nights, the fights over resourcing other projects, the finance manager’s inquest on the overspend … Small wonder margins are low and clients get frustrated.

How autonomation can help

The big batch process doesn’t work. If you’re running rapid projects, you need something else. You need a lean mindset.

If you don’t have time to read The Lean Startup, check out this excellent cheat sheet from Hype Innovation.

He summarises the principle of autonomation by illustrating the ‘power of small batches’:

"The story goes that a guy has to stuff newsletters into envelopes with the help of his two daughters. The children suggest they first fold all the newsletters, then put stamps on every letter, then write the address - do every task one by one. The dad wanted to do it differently, completing every envelope fully before moving onto the next. They competed to see which method was faster.

The dad’s method won, because of the approach known as “single-piece flow”, often used in lean manufacturing. It seems more efficient to repeat the same task over and over, because we assume we’ll get better and faster at it as we go. But individual performance is not as important as the overall performance of the system. Time is lost between the ‘batches’, when you have to restack the letters, and prepare the envelopes. When you consider the whole process as one single batch, you improve efficiency."

Looked at this way, it’s easy to see how the ‘big batch’ agency process has weakness designed in: role specialisation makes too many hand-offs and points of potential failure; there is no consistent oversight across the stages; and you frequently get to the end before finding a mistake you made at the start.

Autonomation empowers a worker - with good technology and process - to own a project from start to finish (single piece flow).

The ‘big batch’ workflow has been applied to small research projects for the last 15 years or so. It’s time to kill it.

How to build a better future

Modern insight platforms have engineered out the need for so many specialist teams, made quality control much easier and empowered insight professionals to work smarter and faster.

So what should a modern project process look like?

To start with, you need the right skillset and the right tools:


For everything except the most complex projects, there should be no distinction between a researcher and a project manager. With the right attitude, training and technology, one competent Insight Planner can comfortably handle modern projects end-to-end.


An insight platform that includes (as a minimum) survey design, libraries, project management, real time analytics and data visualisation; and, ideally, connection via API to an integrated sample platform.

And here’s how it should work:

  1. Insight Planner and Client agree the brief.
  2. Insight Planner designs the survey in the platform [this is the single version of the truth].
  3. Client signs off survey online [if they won’t / can’t, then go old school and export it to a document for them].
  4. Insight Planner floods survey with test data to make sure there are no design errors.
  5. Insight Planner uses test data to build hypotheses for stories, simulate analytics and create visualisations.
  6. Insight Planner selects the sample parameters, sets quotas and launches study.
  7. Study completes in field.
  8. 80% of analytics and visualisations are completed automatically.
  9. Insight Planner completes final 20% of analytics, visualisations and story.

This is insight autonomation in action.

For Toyota, an empowered, multi-skilled operator owns a process end-to-end and supervises several machines.

For agencies, the principle is no different: an empowered, multi-skilled insight planner owns the process and the deliverable. Moving to this sort of model will drive up quality and shorten timescales; it will also make agencies more cost efficient for clients and keep more margin for themselves.

So how do you get there?

For many agencies, autonomation will be a big transition: adopting lean principles, hiring and training differently, investing in technology. It’s a big commitment - emotionally, culturally, financially. But a ‘no change’ strategy will be an even bigger burden.

Here are five key steps to making it happen.

1. Targets

First, agree what you want to achieve. Set some KPIs with monetary values and some milestones. Reduce your outsourced spending? Shorten average project durations? Increase project margins? Increase client or team satisfaction? Whatever your goals, make sure you have a clear hierarchy and realistic timescale.

Then build a rough business case with the costs (people and technology, below) and the benefits you want to achieve. Make sure you have plenty of wiggle room. You can refine it later.

2. Ownership

Appoint or hire someone to lead the transformation and drive long term operational excellence. This will need to be someone with good experience in both professional services management and technology.

You might find someone with experience in the insight business; but don’t make that a deal breaker. As long as you can support them properly with an insider, you can make it work. This can be an interim position to start with - say a contract role to help define and manage the transformation - but that’s not ideal.

Really you want a full timer who can implement and operate; that way you can incentivise on long term business impact.

3. Priority

Identify all the major choke points in your current process. Where do things bottleneck? Where do things most often go wrong? What do people blame for late / costly delivery?

Don’t just do this top-down. Conduct a thorough review with input from all across the agency. Include new joiners, interns, admin staff, clients. Make sure you surface all the potential challenges. Then attach a time / cost / risk premium to each.

Use whatever data you have (timesheets, supplier costs, internal budgets). Prioritise the most urgent items, and refine your rough business case.

4. Platforms

Make sure you build the right spec and choose the right tools. These will be critical investments. You're going to mandate that everyone uses the same research, analysis, visualisation, sample and project management tools. Make sure you adopt wisely. Run a beauty parade. Score them all in a structured way. Don’t rush it.

5. People

Define the roles. Insight planners will need more of a renaissance skillset: strong digital competencies, good commercial aptitude, process management, storytelling, empathy. Hire and train for the the right behaviours as well as the right skills.

Don’t swallow the lie that broad skillsets are impossible to find: today’s equivalent roles in Digital Marketing, Business Intelligence and Customer Success all have similarly broad expectations. Be prepared to lose and excite members of your existing team in equal measure. Actively manage the culture change this will bring. And don’t lose your nerve.

Expect the initial transformation to take between 12 and 24 months. You’ll know when you get there. Your old process will feel as outmoded as sending written documents to a typing pool, dictating emails to a secretary or carrying an A to Z around London.

You will have achieved autonomation, and you will have found the winning formula for adapting to automation.

Then your next job will be figuring out how to implement Kaizen

Leave a Comment

Scroll to Top