5 steps to spot fake AI - as a business person
Fake AI is everywhere and could be a danger to your business decisions. This blog article reveals how to spot an AI impostor.
AI is changing everything - from cars to grocery retail, it will bring improvements at a scale we have never seen before. Most CEOs and Managers understand this and try to leverage that fact in order to get ahead of the curve in their market.
First of all, what is AI? There are many definitions out there for specific AI (and that is part of the problem), but this defintion hits the nail on the head:
Useful AI implies using neural networks, machine, transfer, reinforcement and deep learning in production models… Useful AI eliminates the need for human intervention. Says Daniel Molano, CEO & Co-Founder of Adext
Creating AI is not an easy task. There is no magic software that you download and suddenly an AI emerges. It takes hard work from highly skilled people. This kind of talent is very rare - usually good AI engineers graduated in the top 1% of their class and have already dedicated several years of their lives to machine learning, programming, statistics, and solving complex tasks.
When I visited the NRF Big Show (the major trade show in retail technology) in New York this year, every second company claimed to offer some kind of AI enabled service. Compared to the year before the number of companies claiming this at least doubled. But where do suddenly all the engineers come from to create this mass of AI-enabled services?
The answer is simple and disturbing at the same time - most of these companies did not create any AI nor are they planning to do. From many high level conversations, I have strong evidence to suspect that it comes down to the the following (for retail tech).
- 90% of the companies have nothing. Zero. It is all marketing without any backing. They may have some heuristic or inferential statistics in place. But nothing that would qualify as AI.
- 8% of the companies have very simplistic machine learning approaches in production which are hardly any better than the inferential statistics they have been using for the last 20 years and called it “big data analytics” until 2017
- 1-2% of the companies have actually something that qualifies as AI. The degree of sophistication may differ, but they are actually using valid technology and outperform past solutions quite significantly.
In conversation with clients (usually major retailers) I and my team at SO1 get asked how they should distinguish the real thing from the impostors when they pitch their solution. We recommend the following step-by-step:
- You potential partner builds AI? Well then they need AI / machine learning engineers (and good ones!). Head over to their LinkedIn profile and see how many people with deep AI skills (several years of experience) they employ. Less than 3 will not work - not even for a start-up.
- They also need to recruit a lot in this field. Because the demand for these engineers is high they will be lured into joining other companies all the time. So visit their career page - any sight of ads for machine learning engineers?
- Ask their CTO in a meeting which scientific machine learning paper they read in the past weeks and could recommend to you. You don’t need to read it but if he/she cannot name one right away you can be very sure you just caught an impostor.
- Run an A/B test against your current solution. The AI solution should significantly outperform anything you have today and should do so repeatedly without human intervention.
- Ask for performance-based pricing (regardless if that is actually your preferred mode). If they are happy to do it and you checked all the other boxes: Congratulations: the chances are good that you just found a real AI tech supplier!