Skip to content
Neelima Misra11 apr 20183 min read

How to build your AI on a solid data foundation

Thanks to Artificial Intelligence (AI) and its promise to transform the creative process of humans by augmenting and assisting it with smarter tools we are in an era where every business will eventually be intelligence business. However, many of us struggle to understand how artificial intelligence, big data and analytics are related and how they are going to coexist.

To succeed, an AI strategy must be part of an overarching business plan. Businesses need their AI applications to take advantage of their data to learn about and improve their past performance. Understanding the AI and its Data ecosystem is key to Intelligent Business. This is why I felt the urge to present 5 tips for you today:

1. Know your AI

We use AI daily. SiriGoogle Now, and Cortana are obvious examples of artificial intelligence, but AI is actually all around us. It can be found in vacuum cleaners, cars, lawnmowers, video games, Hollywood special effects, e-commerce software, medical research and international finance markets. Hence, John McCarthy, who originally coined the term “artificial intelligence” said: “As soon as it works, no-one calls it AI anymore.”

It’s necessary for businesses and enterprises to know their AI use cases and the scale in which they are planning use it. Clarity on your AI use cases contributes to the understanding of what kind of infrastructure and ecosystem you need build around it. 

Depending on the use cases, AI infrastructure requirements and data processing capabilities may vary widely from enterprise to enterprise.

For example, predictive maintenance of cranes in remote construction sites, would require the manufacturer to harness machine learning to control rising equipment maintenance costs. The machine learning algorithms in this case will combine data to make a prediction of potential failures or maintenance needs.

2. Understand what data will drive your AI

In the above example of predictive maintenance use case, data from IoT sources (devices, sensors etc), fault reports, audit reports, environmental sources and operator’s history logs needs to be combined in a specific way to identify a pattern.

The next step is to employ advanced algorithms and machine learning to act from real-time insights. In this example to achieve higher accuracy in prediction, an ecosystem of AI, Machine Learning, Data and advance analytics should be created and deployed. The volume and complexity of data may vary based on the type of AI. Many different algorithms need to be tried and tested before the right one is deployed.

3. Find the right data infrastructure for your AI foundation

AI may need the following enabler to succeed:

  • Exponential increase in computational power
  • Availability of low-cost and highly reliable large-scale memory devices
  • Machine learning from actual data sets, not just sample data
  • Voice and image processing algorithms
  • Open-source programming languages and platforms

The traditional data platforms (relational databases and other traditional data systems) may not be able to cater all or many of these requirements. Hence, identifying the right data infrastructure (big data, data lakes and others modern data platforms) is key to success of your AI strategy.

It is worthwhile to evaluate your technology options and data growth plan before making heavy investments on AI. 

4. Train, train and train

The cause of poor performance advance algorithms is either overfitting or underfitting the data. In simple words your machine has the power to learn too much or too less from the data sets given to it.

A famous example of overfitting is the conclusion by AI engine that player performs poorly after being on the cover of Sports Illustrated magazine. Again, this could be true. But it is more likely due to the fact that every sportsperson is bound to have few highs and lows in his/her performance throughout his/her career.

Striking the right balance and training your algorithms to deliver the right level of accuracy for your AI use cases is a very core to the data science capability you should build in-house.

5. Revitalize your data ecosystem to ensure your AI remains Intelligent for future

To ensure your algorithms stay accurate they must be continuously updated with new training data. Better algorithms mean higher accuracy, which in effect increases the amount of work that can be automated. Ultimately AI is aware but not yet self-aware — and only aware of things we input. Machines may be learning fast, but they’re only as good the parameters we set for them.

To ensure that your AI is accurate in future, consider revitalizing your data ecosystem regularly.


Neelima Misra

AI, Analytics and RPA Lead