A Behind the Scenes Look at Building an Analytics Platform

One of most memorable scenes in the classic movie Wizard of Oz comes when Toto accidentally pulls back the curtain to reveal an old man frantically working the switches and levers that control the ‘great and powerful Oz.’ What seemed magical to unwitting visitors was built on a foundation of human inspired technology.

As a social media marketer, chances are you employ a variety of tools to support your efforts. There are platforms for ‘listening’ to what people say about your brand, and others that optimize content publishing efficiency. We built a competitive intelligence platform that enables brands to ‘see’ the best performing content and campaigns across social networks. But before we could present a finished product that delivers insights in a magical way, our team of data experts followed a multi-step process that goes into building a complex analytics platform. Let’s take a peek behind the curtain.

Gathering Data

The first step to building an analytics platform is gathering data. This is obviously the most important step, as without data, there’s nothing to analyze. All of the data that’s gathered is in its raw form. In our case of building a social media analytics platform, we started by collecting available data from the APIs of major social networks including Facebook, Twitter, YouTube, Instagram, Pinterest and LinkedIn. The APIs provided by the social networks allowed us to seamlessly integrate their data into our application. One of the major challenges of this step was dealing with the sheer volume of data streaming in on a daily basis. We had to build and plan for the storage and modelling of all the data to make constant updating easy.

Processing Data

Once we built the necessary infrastructure to gather and store the massive amount of social media data that we planned to include in our finished product, we then moved on to processing the data. This is really the meat of any analytics platform and focuses on taking data in its raw form and turning it into relevant information for the end user. During the zenith of ‘big data’ hype, companies focused almost exclusively on the volume, velocity and variety of data. While these are all important, the true value of any large data set lies with the actionable insights it provides. For our social media analytics platform, some of the elements that went into processing the data to make it useful included:

• Natural Language Processing (NLP): To glean insights that are valuable, we utilized several NLP algorithms that help us gain a deeper understanding of social media content and its nuances.
• Sentiment analysis algorithms that help understand the conversations that are taking place between the brands and their consumers
• Creating and populating our search engines that help users search and discover content easily
• Modeling the database and preprocessing data so that data can be retrieved faster by the end user

Depending on what type of data your platform analyzes, there can be limitations to what the technology can recognize. In the case of social media data, we found that certain things such as identifying social campaigns across channels is best left to humans. Therefore, our data processing system includes a combination of robust algorithms and human analysis.

Analytics Platforms

The third step in creating an analytics platform taking all of the processed data and creating what users actually see. A good data analytics platform should be created in a way that allows users to slice and dice the data in a way that works for them. It’s critical to create and design the user interface in a way that allows users to navigate, access and analyze data quickly, clearly and visually. A lot of thought went into visualizing the data in the form of charts and graphs. As the saying goes, “a picture is worth a thousand words.” In the case of an analytics platform, a picture is worth thousands of rows of data. However, it is very important to understand users’ needs and offer the right kind of visuals that help them gain insights from the underlying data.

The process doesn’t end here. All analytics platforms must be constantly worked on, tweaked and improved as data changes, client needs shift and technology gets better. The basic steps to building an analytics platform manifest in platforms such as Unmetric. However, the more that end users like you use a platform, the smarter we become about the best ways to gather, process and present data that leads to informed business decisions across the entire organization.