How External Data Can Elevate Data Analytics and Machine Learning To New Heights

Data is valuable. It helps uncover valuable insights, understand the customer experience, analyze their exposure to risk, and assess the success of evolving business models. Ultimately, the right data helps drive better business decisions. 

A few months ago, we hosted a session at the MIT CDO IQ 2021 Symposium. In this session, Mohammed Aaser, Chief Data Officer at McKinsey & Company, and Ajay Khanna, CMO at Explorium, shared insights on the ways that companies can take advantage of external data. You can click the link below to watch the full video, and read on for a summary of what was discussed. 

The Value of External Data

Accurate predictive modeling and comprehensive competitive analysis can be the difference between a company that differentiates itself, and one that falls behind the leaders in the marketplace.

Business leaders understand the value of external data. Increasingly, organizations are encountering the limitations of internal data when used to power data analysis and machine learning models. In many cases, companies are building models based on their own data – available data from their internal sources – rather than looking beyond their four walls to find the most relevant external data sets for their particular use case.

Enriching internal data with relevant external data signals can help power improvements in growth, productivity, and risk management.

Relying on traditional internal data sources doesn’t provide the entire picture as they are often limited in scope. For example, an eCommerce company may have data on their customers, but not on prospects. The data they have on their customers is typically limited to touch-points on their website, clicks, and form fills, and might not include other relevant demographic data that could be used to optimize segmentation and marketing campaigns. 

With the use of external data, companies are able to build more accurate predictive models to fuel more effective, data-driven decision making processes. 

Some examples of external data are: 

• Geospatial and satellite data, such as foot traffic, point of interest, and real estate
• Private business data, such as revenues, headcount, location, and technographic data
• Social media data, to obtain audience insights and gauge sentiment 
• Consumer data, such as transactions, receipt data, search trends, and census data
• Web-harvested and online data, such as web traffic, online reviews, and digital app metrics
• Weather data, such as temperature, precipitation, storms, and other real-time forecasts
• News, IP, and legal data, such as patents filed, legal actions, and research journal feeds
• Public data, such as federal, state, and local filings; and macroeconomic indicators
• Industry-specific data, such as trade flows and shipping, travel bookings, healthcare claims, and agricultural metrics
• Pandemic recovery signals

External data can be used for:

• Customer analytics; analyzing prospects and identifying behaviors
• New product research
• Strategic analysis, understanding competitive trends and product improvement opportunities
• Operations and forecasting;  demand forecasting and predicting the growth of different customer segments
• Risk management, reducing your operational, supply chain, and reputational risks

The data universe is growing rapidly, with thousands of big data sources available today. External data has the power to deliver a meaningful and substantial impact for any organization. 

Hedge funds have been particularly proactive in incorporating different types of data into forecasting models in order to make smarter investment decisions. According to Mohammed Aaser, half of the top 100 firms are now including external data in their models, such as consumer spending data, or other information openly available on the internet.

How to get started

Typically, the external data acquisition process is time consuming. It’s important to go through the appropriate steps to ensure the ROI of a data purchase. 

1) Identify an anchor use case that can benefit from external data

Focus your efforts by selecting a use case for which your internal data is insufficient. Sharing these early use cases with senior management can also help you build momentum and drive investment in your work.

2) Establish the key roles

Define the team needed to carry out your external data work. Typically, this will include:

• Data scouts or strategists, who will take charge of the sourcing process
• Data scientists and analysts, who will apply the data to use cases
• Data engineers, who will ingest, prepare, and use the data
• Architects and DevOps engineers, who will develop platforms and manage access to data
• Data review panels, who will review the use and risk of data collection
• Purchasing experts, who will contract, license, and negotiate with data providers

3) Leverage external data platforms to accelerate the data acquisition process

External data platforms can dramatically speed up your process by alleviating much of the effort typically involved in acquiring and operationalizing external data.

With such a vast ecosystem, it can be difficult to track and manage which data sources you should be using. It’s equally complex to assess and review those sources to justify your investment and build a powerful business case, on top of quality and privacy concerns. 

External data platforms can help streamline your data acquisition process with:

• Automated data discovery
• Access to thousands of data signals relevant to your use case
• Integration of the new data signals with your internal data
• Automated feature generation
• High quality and compliant data

Ensuring you get enough value from your external data sources is key. With a data marketplace, it is difficult to understand ROI prior to purchasing a new dataset. With an external data platform, you shouldn’t have to worry about purchasing a dataset that turns out to be of no use. 

Learn more about the Explorium External Data Platform.

It’s time to unlock the true power of external data

The adoption of external data for in-depth analysis is growing rapidly. Chief data officers are leading the way, bringing external data on board as part of their companies’ overall data strategy. 

Watch the full MIT CDO IQ session to learn more about external data, and explore solutions for some of the challenges restricting external data acquisition.