DataSift Takes Big Data Analytics of Social Networks to the Next Level

We all know data is growing at an astronomical pace. By many accounts, the sheer amount of data coming at us is overwhelming. According to one survey, enterprises today create only 8% of the entire data set they consume. That leaves quite a bit of external data to collect and process. Increasingly, for many this data is coming from Social Media. The concept of Big Data does address these large, growing datasets. However, challenges still await the social media data landscape.

Enter DataSift, a UK-based startup launched in November 2011. Since then, they’ve covered quite a bit of ground and just secured an additional $7.2M this week. I had the opportunity to sit down with Rob Bailey, CEO of DataSift at the Under The Radar conference last week to discuss the company and their value proposition. Rob confirmed that “data is exploding” and while many companies are able to track sentiment of social networks, DataSift is able to provide the “metadata of social data platforms through the aggregation of social data platforms.” Simply screen-scraping data from social networks provides one level of value to users. By using “augmentations of the data using geo, sentiment, meaning and context”, DataSift is able to provide a much richer context to clients. It is this very metadata or relationships between data, over time that starts to get interesting.

According to Rob, the challenge facing DataSift is people understanding the space. Bringing different data elements and considering the metadata of social networks is an exercise in thought. Integration of massive data streams and context on the data present an interesting challenge. These challenges could be addressed over time with adequate resources. However, providing trending information from the variety of data social media sources, with context, in real-time is valuable.

Analytics of data performed by data scientists is one approach. For everyone else, gaining ready access to valuable data sources and making sense of them in real-time is one area to watch very closely.