Why TSAPI Will Save the Survey Industry and What You Can Do to Help
By TSAPI
- article
- Integrated Data
- Survey Software
- APIs
- Data Connectivity
- Survey Data Centralisation
- Survey Research
Why why why…TSAPI?
Do you think insights professionals need to stay relevant in times of great change? Ever had a gutfeel our sector is falling behind a little? Would you like to see more innovation, fewer bottle necks, and less of the hidden technical debt which we’re all paying for? Perhaps you’re not averse to supporting a positive initiative that will be a shot in the arm for our industry? Please read on…and demand TSAPI!
Demand what now?
With all the AI disruption happening it’s imperative we get our human insights data into a universal format, ready to connect, TSAPI is that format. TSAPI is the only industry Standard for Transferring Survey data via API. It’s an open-source (free) format of data transfer which any platform can offer. Like they do for different file types. Wider adoption will enable everyone to transfer survey data into any platform, as easily as sending emails. Oh, and in doing so connect our sector to the valuable heart of the global knowledge economy. We just need platform users, clients, and insights teams, to demand that connection from their suppliers.
Slow down there tiger, what is an API?
Think of an API like a messenger between different computer programs. It helps them talk to each other and share information. For instance, when you send an email from one service to another, like from Outlook to Gmail, you don’t need to worry about the technical details of how it happens. The API ensures that the message gets through smoothly.
How?
Humans! A long time ago in a pub far far away, some humans got together and agreed an API Standard for emails. An ‘API Standard’ is like a common language. The documentation is like a phrasebook. When computer programs understand the same language, they can talk to each other. And thank goodness they did hey? Imagine having to manually convert an email from Outlook’s format, into Gmail’s format, then import it, just to read the text. Then do it all again in reverse before you could send a reply. Imagine…
That sounds pretty old school, we’ve got one of those for survey data right?
Nope. Currently we’re like 1000 people in a market square, all trying to say the same thing, but using 1000 different languages, that we all made up ourselves. Some of those languages are basic, some complex, some communicate well, some get things very wrong. Confusion reigns. What’s worse, crowds of people are arriving from all directions, each with another unique language. It is, at best, embarrassing. Seriously though, this situation is fast becoming a significant sector-wide risk.
Could you get a little more technical? Asking for a friend.
Indeed, let’s discuss the specific challenge – conversion. Platform A has its own API definition for transferring survey data. Platform B has its own API definition for transferring survey data. The survey data is fundamentally the same e.g., single response (SR) and multiple response (MR). However, naming conventions of each API were defined independently. They do not match.
Therefore, we cannot transfer survey data from Platform A to Platform B.
Developers are excellent problem solvers! They set about building a converter. Imagine the line as the converter.
However clever Trevor, this is still a hack. Converters take time and money. They are prone to error and need maintaining. If either platform changes its definition, the converter may stop working. Two-way flow may double the resources required.
If there were only two platforms in the whole world, problem solved. In reality, this is not a scalable practice.
You mentioned sector-wide risk? Like the impact of scaling technical debt or something?
That’s exactly it, you catch on quick! The number of platforms which collect, process, analyse and/or visualise survey data is on the rise. Nearly all need to transfer survey data. This leads to a big problem.
Imagine each node below is a survey data platform, each with a unique API definition. Imagine each line is a converter, developed by a human, to match-up the platforms’ terminology.
If all 14 platforms want to flow survey data between each other, a total of 91 connectors need to be developed and maintained across the industry. Of course, there’s poetic licence in the above image, not every platform in today’s MR world must connect to every other platform. The point made here is, in many cases, software developers in our industry are being distracted from their own product innovation, due to the vast amount of time spent on connectors. Looking down from an industry level, the ripple effect of this vast amount of technical debt is being felt across the sector. We all trade on quality and credibility, yet some of our technical foundations are fast becoming unstable.
Not only that, when the booming numbers of external innovators ask, “how do we plug in to your sector’s data?”, then realise the work involved in servicing platform-specific API definitions, they shift away from our industry.
Today’s world is a sector-on-sector competitive market, and we’re losing out. We have such value to offer but if we can’t find a solution soon, we’ll be someone else’s lunch.
Gulp. So how do we solve the puzzle?
Humans. We simply define, distribute, and maintain a standard (a shared phrase book) for Transferring Survey data via API. That work has already been done for you, by the team of TSAPI volunteers. The definition has arisen independently out of 3yrs of international consultation, and cooperation, among multiple software providers and survey data users. All resources and documentation are released under creative commons licence 4.0. Meaning they are, and will be, free-to-all, forever. www.tsapi.net
Phew! So, the future’s bright then right?
Could be. As Marc Ryan writes, “The freedom to move data into a processing platform of choice is something that unlocks the power of survey data and will drive innovation in tools supporting survey data sets.”. (See: How a Lack of Standards is Killing Survey Research). From collection to analysis, visualisation, and insight – the possibilities are endless. In the same article Marc highlight TSAPI as a glimmer of hope which, “…raises the bar for how the survey research industry should operate.”.
This is a time of incredible technological advancement, and we have an opportunity to connect into that world. Survey data is human insight. We need to enable our sector’s value to be involved and appreciated.
What can I do?
We have, in effect, taken the horse to the water. We need you to help encourage the refreshing consumption of cool crisp survey data API standards! The TSAPI concept will only be realised if the survey data platforms we all use offer that connection. It’s no big thing, they already offer lots of connection formats which are not their own – think .sav or .csv downloads. For that to happen, we need you to demand TSAPI from them.
Hang on, were you born yesterday?
The standard draws on many decades of survey data implementation and execution. The concept of a Standard was introduced on the 6th August 2020, globally. In APAC at the Australian Research Society Conference. Then Europe via the ASC’s Online Series, the ICG, and ResearchLive magazine. Then on US time via Quirks Media and mrweb.
The definition has arisen independently out of 3 years of consultation, and cooperation, among multiple software providers and survey data users. TSAPI is an independent, non-profit initiative. The resources are, and always will be, free to all, forever. All we ask is that the TSAPI Initiative is credited (using tsapi.net) and that the Standard is implemented correctly.
How are your grades?
Exemplary! In fact, TSAPI has been endorsed by the Association for Survey Computing. The ASC is the World’s leading community for anyone who works with surveys, survey data and insights. A Community Interest Company (CIC), with roots extending back over 40 years. The ASC conducted technical due diligence and are subsequently championing the cause, for which we are eternally grateful. Here’s a snippet of that endorsement, “We view our endorsement of TSAPI as a way to
encourage adoption of this standard and to facilitate further discussion across the industry that may help to develop the standard and give it the attention we believe it deserves.” Read in full here: https://ascconference.org/tsapi-survey-data-interoperability-standard/
TSAPI was a named finalist in The Research Society’s Award for Technology and Innovation 2023. We’re in discussion with ESOMAR’s Professional Standards Committee in the hope they will also endorse and promote TSAPI.
In a nutshell, what are your main objectives?
- Help preserve & develop survey data’s position in the knowledge economy.
- Improve connectivity & interoperability.
- Facilitate innovation.
- Ease the connection of MR insights to other industries & sectors.
The benefits of TSAPI will be far-reaching. Evaporating developer frustrations and insight bottle necks. Improving quality, consistency, ROI, and efficiency. Facilitating innovation to help insights and analytics professionals stay relevant in a time of great change.
Please do support TSAPI – we need all the help we can get.
Author
