Data Science is often looked at as a game that the big companies have mastered and smaller companies can only watch form the sidelines. It is a game that is too expensive, too difficult, too time consuming for any smaller players to excel. This has led to the numerous articles and opinion pieces saying that the majority of data science projects never make it into production, that data science doesn’t provide value or that big data is dead. However, I believe there is a bigger question to ask. Why are so many companies struggling to utilize their data and find value in it?
I believe the answer is more often tied to the availability of data as well as the communication between departments. I have seen large government agencies fail at data because data availability and consistency was so siloed between departments. On the other hand, I’ve seen tiny nonprofits succeed because there were only one or two people in charge of the data and so it was accessible and consistent.
The 1% problem in data science is often not a question of resources, it is a question of accessibility.
A company doesn't necessarily need a large budget or a huge data science team to start getting insights from their data. With the right tools and/ or the right partner, you can begin getting data insights and acting on those insights quickly. Often the most valuable part of working with a data partner is getting your data into an easy-to-read, accessible format.
Once data processes have been created and executed, the next question is what to do with that information. Small companies often have a few people who have been deeply entrenched in the business for years. This institutional knowledge is a great starting point for what kinds of questions to ask of your data, starting with what the team “knows” to be true. This can lead to deeper insights and understanding of years of observation: when these observations hold true, when they don’t and, most importantly, is there data that supports that there are new trends emerging and that long held company truths are changing and evolving.
Many small and startup companies feel that being data-driven is not for them. Not because they don’t think it is helpful (many companies are focusing on analytics, 91% of companies say they are accelerating investment in data and analytics according to a NewVantage study) but because they think it is impossible given their budget, resource and time constraints. Focusing making your data accessible and answering one question at a time will get your organization well on their way to being data driven without the huge costs and headache.