Over the last year, 7 out of 10 businesses raised their AI technology investments, but finding the right talent to fuel business growth is far more difficult.
As businesses turn to artificial intelligence (AI) technology to help reduce costs, improve productivity, and gain value from their data, machine learning engineers are becoming a more valuable resource. According to a recent RELX survey, 63 percent of businesses said AI had a positive effect on their ability to remain resilient during the pandemic, and about 7 out of 10 businesses have increased their AI technology investments in the last year.
However, as with any new innovation, bringing the promised value to life relies heavily on getting the right people in a position to help. And top AI talent is scarce—nearly two-fifths of businesses mention a shortage of technological knowledge as a major barrier to implementing AI.
The demand for AI skills outnumbers the supply.
One of the most serious issues currently confronting businesses is that the demand for AI positions far outnumbers the number of eligible applicants. Indeed.com data reveals that there are three times more work listings for AI-related roles than job searches and that postings for AI jobs have risen up to 12 times faster than searches for AI jobs between 2016 and 2018.
Machine learning as a profession has also grown in popularity. In January 2020, Google searches for ML engineering work in the United States reached an all-time high.
Other search words, such as “how to become a machine learning engineer” and “machine learning engineer salaries,” have grown by more than 5,000 percent in the same time span.
Despite the interest and demand, there is a scarcity of relevant skills, expertise, and experience. According to the Deloitte’s State of AI in the Enterprise survey, there are around 300,000 AI practitioners worldwide and fewer than 40,000 top-tier AI experts on LinkedIn. It’s no wonder that AI developers and engineers are in high demand at companies adopting AI, regardless of whether they’re just getting started or have a lot of experience.
Given the limited global talent pool, what distinguishes an outstanding machine learning engineer from a competent one? What would you do to help these talented people succeed if you are willing to recruit them?
How do you choose the best machine learning applicants for the job?
As you would expect, we know a thing or two about recruiting exceptional machine learning talent at Google, but we’re up against startups and organizations at the cutting edge of AI science. What makes the best so amazing, we asked some of our engineering managers. Here are the top attributes that all Google Machine Learning engineers have in common, as well as what you can look for when recruiting them:
They are well-versed in the design of distributed systems. Huge datasets, parallel data processing, and distributed training are all requirements for machine learning workloads. Knowing how to use storage, network, and compute resources efficiently will dramatically speed up the process and lower costs.
They break down machine learning solutions into a modular architecture. Modularity is critical when implementing machine learning systems, just as it is in software development. It not only allows team members to work effectively on individual pieces, but it also promotes reusability for potential projects or other teams.
They understand the importance of research. From validating input data to model output to integration code, a top-tier ML engineer would prioritize rigorous testing at each level of the ML development process.
They are still concerned about security. Due to automation, large amounts of complex data, and handling and processing workloads in the cloud, AI and ML introduce a specific collection of security threats. ML engineers must approach ML production with a security mindset, ensuring that all training data, software, and communication channels are properly protected and controlled.
They have excellent communication skills. Since machine learning engineers work at the crossroads of many disciplines, efficient communication is critical. They’ll have to justify everything from rank to risks to trade-offs to a variety of audiences, including data scientists, developers, administrators, and business users.
Great machine learning engineers take into account the complexities of advanced technology. Average ML engineers are distinguished from excellent ML engineers by their passion for the diversity of feedback and their ability to cultivate a supportive community.
They know what “good enough” means. In an ML system, there are many aspects that can be changed and automated, but it’s critical to be able to identify when the effort outweighs the benefit. Great machine learning engineers keep their eyes on the project’s goals and priorities, and they know when to call it a day. They will also get a better return on investment by moving on to the next iteration rather than taking the time required to perfectly match the current model.
They express themselves clearly. Similarly, when an ML engineer requires internal-focused resources and efficiency improvements, they should speak up. The company still needs more functionality as soon as possible, but it often lacks the means and processes to make it happen. A single ML model takes the majority of models 1-11 weeks to launch, and 26% of practitioners agree that delays are triggered by a lack of executive buy-in. ML engineers will be expected to make difficult requests for investments in areas that will not pay off immediately but will enable them to be more profitable in the long run.
They are adaptable. Machine learning projects may run into a variety of issues, such as obtaining enough data or developing models that aren’t accurate enough to satisfy business requirements. To be able to execute projects, you must be able to change strategies quickly to solve challenges without being overwhelmed or losing sight of the end goal. Tooling versatility that has been shown is also a plus. Another indication of top talent is experienced with multiple frameworks, such as TensorFlow, PyTorch, and scikit-learn.
They are inquisitive and resourceful problem solvers. Things will inevitably go wrong, and the best ML engineers will have to think outside the box to solve problems involving machine learning, data, and apps. Sometimes, what appears to be a data science issue (false positives) is actually a subtle issue further upstream in the machine learning pipeline that is causing bad performance. A good machine learning engineer must be comfortable exploring a wide range of potential root causes and have the patience to keep asking questions.
They’re excellent mentors. Since there are so many possibilities for advancement in this emerging area, a great machine learning engineer will help others by sharing their unique insight and experience. Their experience dealing with complex processes and a wide range of stakeholders makes them a valuable resource within the company.
They are modest in their approach. Our world is not static, and AI is evolving at a rapid pace. We try to note that we are still learning and that while certifying a product as perfect is an unattainable objective, we can always improve. As an example, when Google removed gender labels from our Cloud Vision API, ML practitioners had to make tough decisions over time. Internally and externally, these changes can be difficult to explain, but great machine learning engineers respect the complexities of advanced technology. Average ML engineers are distinguished from excellent ML engineers by their passion for the diversity of feedback and their ability to cultivate a supportive community.
Even within engineering communities steeped in data science, we’ve discovered that machine learning is often a new ability for many. According to a recent survey conducted by Kaggle, the world’s largest group of data scientists and machine learning professionals, slightly more than 55% of data scientists have less than three years of ML experience. Just about 6% of licenced data scientists have been using machine learning for a decade or more.
Recruiting and hiring outstanding machine learning talent is difficult, but not impossible, given the limited global talent pool. While learning how to spot talent and looking for the characteristics mentioned above will help businesses hire high-quality ML engineers, they also have options when it comes to platforms and tooling.
Many of the scalability, stability, and developer velocity issues can be solved with tools like Google’s Cloud AI Platform. When you combine great talent with great tooling, your workers will be able to excel and add value to their machine learning ventures.
Keep in mind that artificial intelligence is a team sport!
Getting the best machine learning engineers isn’t the only criterion for AI performance. According to a Rackspace survey from 2021, only 17% of respondents claim they have mature AI and ML capabilities and a model factory framework in place. Furthermore, the majority of respondents (82%) said they are either figuring out how to apply AI or are having difficulty operationalizing AI and ML models.
Effective AI projects necessitate input from people who aren’t engineers. Since they lack a common vision outside of engineering, the majority of AI ventures (87 percent, according to VentureBeat) never make it into production. Although engineering expertise will still be important in AI, it’s critical that businesses create workflows that enable everyone—technical and non-technical alike—to participate in moving projects from test to deployed AI.