This #TechTuesday, Learn How to Transform Big Data into Smart Data

shutterstock_225917539.jpg

It’s no use gathering a ton of data but not knowing how to harness it. Here’s how to transform big data into smart data. By Dr. Yan Qu (Vice President of Data Science, ShareThis)

Social networks like Facebook and Twitter have radically transformed how we find, consume and share information. Online sharing has fundamentally altered and redefined every element of communications. All of our online social activity generates data about our likes, dislikes, interests, feelings and much more.

As consumers continue to share more content across channels and devices, we need to move beyond the “big data is here” narrative and start figuring out how to find the value in it. Big data is good, but smart data is better.

So how does one go from big data to smart, actionable data? One needs to think about three questions:

  • Why (priorities)
  • What (both type of data and the resources to collect it)
  • How (processes)

Smart Data Starts with Why

Smart data is relevant data.  Why do brands want to tap into social data? They want to better understand their consumers, better track their competitors and market conditions and better tap into consumers’ future purchase decisions.

Relevant data like these can help brands make data-driven decisions to engage their consumers and maintain the brand’s competitive advantage.

What Data do we Need?

Big data is large in scale, messy and unstructured. In order to utilize the data, it needs to be captured, cleaned, structured, organized and possibly enriched for later analysis, which requires considerable resources. Because of this, it is important to apply business objectives to guide what data to collect and process.

How we Process Data

For example, ShareThis aims to provide brands a comprehensive view of consumer sharing behaviors to drive brand engagement. To achieve this, we have tapped into our network of 3 million publisher sites and apps that use our sharing solutions.

We collect users' page views and sharing data up to several billion transactions per day. The data is further enriched so that we know what topics or brands users are sharing about, and what their sentiment is towards these topics or brands.

With the growth of mobile, users constantly engage across multiple devices to do business, get information, socialize with their friends and entertain themselves. To capture user behaviors across multiple screens, we added data streams of mobile apps from third-party data sources and from our own mobile app sharing tools.

Our big data harnessing process has been iterative, driven by the business objective of a unified view of the consumer and the innovations in the consumer digital experiences. New, different data sources often pose new challenges we need to tackle.

Fresh Challenges

For example, in different data sources, consumers may be represented in different forms, e.g., as anonymous cookies in desktop logs, device IDs (e.g., IDFA) in mobile app logs, Twitter handles or Facebook profile names from social networks.

Identifying a user accurately across devices and screens is a critical problem to solve for providing an accurate and comprehensive view of a user’s today digital experiences.

Smart Data and Actionable Insights

The ultimate goal for big data is to draw conclusions and empower companies to make better data-driven decisions to address their business challenges and identify new opportunities.

For example, in a recent Sharing Trending report, we found significant differences in how various events draw engagement from different audiences. International crises (like the threat of ISIS in the middle east) drew more sharing from older and male audiences; while the Ferguson unrest drew generally younger and female audiences.

Brands that want to act on what’s trending and deliver real-time marketing solutions to drive relevance and results can tap into the above insights so they can engage with the right audience, at the right time and in the right context.

So What Now?

Unlocking the value of data for business is a journey. With the right technology, platforms and analysis, we can work to go from big data to smart, actionable data. It’s important to remember the components in the journey – why (priorities), what (resources) and how (processes). The resources and processes serve the why.

What kind of data would be helpful to your business?

Image credit: faithie via Shutterstock.


About the guest blogger:Yan Qu is VP of Data Science and Chief Scientist at ShareThis, where she leads the data science initiatives. She has designed Social Quality Index (SQI) - and unique metric for valuing the social activity that occurs around online content. SQI won the AdWeek Project Isaac Award and has been adopted by major ad agencies for media planning purposes. Yan holds an M.S. in Computational Linguistics and a PhD in Information Language Technologies from Carnegie Mellon University, and an M.S. in Linguistics from Tsinghua University in Beijing.