What is data sourcing and how should companies approach it?

We live in a data-driven world.

A hot topic for years, the global focus on data has only intensified with the artificial intelligence breakthroughs of recent months.

But not all companies are equipped to benefit from the opportunities offered by the dizzying and ever-multiplying amount of available data.

Why? It often comes down to how companies approach, source and handle external data. Coming up with the right strategy is far from easy.

The Neudata platform is ideally positioned to help you find the data you are looking for, with a unique scouting service to help.

But it helps to first have a strategy in place. This article will help organisations consider how to approach the subject of external data and put together efficient processes from the beginning.

Read on to learn about:

• Alternative data or external data?
• What is data sourcing?
• Examples of data sourcing strategies
• To outsource or not?
• Sourcing is important — but not everything
• What is Neudata?

Alternative or external data?

The alternative data market is worth billions and offers a huge range of options to buyers – the Neudata platform features more than 7,500 datasets from almost 2,000 vendors.

That gives a sense of the scale of the data business, which has grown rapidly in recent years, and the mass of datasets now available.

Hedge funds and other institutional investors have been at the forefront of adopting alternative data, but corporations and other organisations are now ramping up their usage.

Their motive is clear: 87% of 914 executives agreed that data is the most important competitive differentiator in the business landscape, according to a survey by The Economist Intelligence Unit, sponsored by Snowflake, a data provider.

As if that wasn’t enough, 40% of respondents said digital transformation was the most critical priority to their organisation’s success over the next three years.

And 83% said that organisations in their industry will be routinely using AI to process data in the next five years.

Corporations often call it external data, not alternative data, to differentiate it from the internal data they already gather.

McKinsey's report — Harnessing the power of external data — provides a helpful introduction into the types of external data available and different use cases for companies.

Neudata also has a collection of blogs and free resources introducing you to the alternative data landscape. This intelligence article highlights some of the approaches and techniques which have attracted more interest, from web-scraping services to natural language processing.

What is data sourcing?

Data sourcing is the process used by organisations to identify external datasets, which can then be adopted into work-flows and – hopefully – yield a commercial benefit.

It helps if organisations have a clear idea of the kind of data they are looking for and which problems they seek to solve – so they can  evaluate the time and money needed to tackle the initial problems associated with its adoption.

Results from a Neudata survey of 45 data providers implied that companies with a specific use case in mind are more likely to adopt external data.

How to find the data? It varies by sector and use case, but word-of-mouth, advertising, industry media and conferences are common routes to finding alternative data, especially in financial services.

For organisations with less experience of the data world, a variety of marketplaces exist with various approaches and fee structures.

Neudata offers a unique data sourcing service which actively helps clients source the right data for their needs.

We don’t buy or sell data, or require data providers to pay us a revenue-share or commission in exchange for recommending their products to data buyers. That means data buyers get unbiased intelligence that’s tailored to their specific research goals and strategies.

Examples of data sourcing strategies

As mentioned in the previous section, companies should have a clear idea of the kind of data they’re looking for before they start seeking out new sources.

But it is just as important that your organisation is aware of the advantages external data offers and the ways in which it can be used.

It is one thing to understand why it is beneficial to invest in external data, and another to have a data culture in a company that can utilise it effectively.

Companies beginning their data journey should be asking:
How to incorporate external data – does it complement the existing data infrastructure or sit outside of it?

• What are the costs – how much budget should be allocated?
• What data can be integrated – and is the technical knowhow in place?

Central command

An important strategic question to ask is whether data adoption is spread throughout the organisation or centralised in the IT department.

The latter is most common, due to the reduced risk of data breaches, faster processing and better overall data governance.

But organisations should consider whether a more decentralised approach would allow data to live closest to those who most need it, as The Economist Intelligence Unit/Snowflake survey found.

How to get started

McKinsey recommends in its report a three stage process:

1. Establish a dedicated team for external data sourcing
2. Develop relationships with data marketplaces and aggregators
3. Prepare the data architecture for new external data streams

This seems like a sensible plan to us. And if your organisation needs help as it begins its external data sourcing journey, why not ask Neudata for help or advice? Our team of experts would be happy to help – just email info@neudata.co as a starting point.

To outsource or not?

Privacy regulations and lack of tech knowhow are some of the most common difficulties companies face when integrating data from varied sources, while doubts over the accuracy or quality of data are common.

These challenges have led to more use of outsourcing, particularly at the start of a company’s journey with external data.

Data collection through custom web-scraping, data integration and data management and storage can all be outsourced to service providers.

Is this a good idea? The Neudata view is that using service providers can allow firms to gradually develop an external data team effectively, by investing in necessary functions for their use cases.

Whether a company outsources or not, a strong internal strategy and dedicated headcount is important. According to a Neudata survey of providers, this function is led by a data strategist, chief data officer or data scientist in most cases.

In its report, McKinsey recommends employing a dedicated data scout/strategist, who then works with data analytics and engineering teams.

This won’t be possible for a lot of firms – and, as mentioned, Neudata’s data scouting service can help – but would be a sensible long-term ambition.

Sourcing is important - but not everything

The focus of this article has been on sourcing data: how to put efficient systems in place to identify the most useful datasets for your organisation’s purposes.

But sourcing isn’t everything – once you have found the right datasets, you need the right data management system in place to onboard and deploy data.

Nasdaq, in collaboration with Wakefield Research, surveyed 200 portfolio managers and found that, even in the sophisticated finance world, there is widespread frustration over data management.

Sixty percent of fundamental managers cited an inability to quickly onboard or deploy new data. Quantitative managers were more satisfied on this basis, but the same proportion (60%) of quants were frustrated by an inability to quickly test new datasets.

If those levels of frustration persist, there could be more outsourcing in this area going forward, especially when it comes to corporations that have less experience with data than institutional investors do.

It should be noted, however, that the frustration felt by relatively sophisticated users of alternative data shows how far the industry has come. Casting the Net, a research paper by hedge fund trade association Aima and technology company SS&C, found that alternative data’s popularity has led to one of its biggest challenges.

“Indeed, the universe of alternative data sets is expanding so quickly that many of them are not going back in time far enough for models to reveal patterns or capture signals.”

Though it’s not perfect, this popularity demonstrates the market growth and plethora of opportunities which will be available to corporations as the amount of data increases.

What is Neudata?

Neudata is an alternative data-focused research platform that specialises in the objective and neutral assessment of  and datasets. We help institutional investors, corporations and leading global organisations find the most relevant alternative data sources to use in their internal data ingestion processes.

Our platform is the global authoritative source for unbiased, independent alternative data intelligence.

Since 2016, we have helped our clients understand the landscape of available datasets, increasing the efficiency of their data spending budgets. Neudata’s data buyer clients represent 60-70% of industry-wide spending on alternative data.

If you are a data provider/owner and want advice on selling or monetising your data assets, or if you’re a potential buyer looking for an introduction to the alternative data landscape, contact info@neudata.co to discover how Neudata can help.