What’s Your Route to Enterprise AI Adoption?
Take a look at the current indicators that favor an in-house or platform-based approach to enterprise AI development, with an eye to emerging trends.
Generally, emerging technologies that are valuable enough to become popular tend to decentralize at the earliest opportunity. From the print bureau to the home printer, the processing lab to the smartphone camera, the mainframe to the personal computer — the phase prior to this consumerization is hallmarked by business services gatekeeping the new technology and meting it out to consumer demand in small and increasingly profitable measures as hardware costs reduce — eventually reducing enough to diffuse the technology and kill the 'gatekeeper' business model.
The explosion of data storage and processing needs over the last twenty years has not only kept this from happening in the business information services sector, but has, according to some sources, practically eradicated the in-house data center1 in favor of the cloud.
Now, under pressure from the latest iteration of the AI hype machine, the explosive interest in machine learning and intelligent process automation once again raises the question of where the market for a hot new technology will ultimately settle — in external provisioning or in-house?
In the case of machine learning, the core model is very different from general cloud provisioning, in that the data is still voluminous but the enabling technologies are free, relatively lightweight, and practically immune to the forces of market capture.
However, the critical differentiator is the current need for expensive computing resources in model development and updating — a factor that promises to change as more lightweight machine learning systems such as Apple Core ML evolve, new players enter the AI ASIC market, and Nvidia's acquisition of Arm promises to democratize access to GPU-accelerated AI.
In this article, we'll take a look at some of the current indicators that favor an in-house or platform-based approach to enterprise AI development and deployment, with an eye to emerging trends.
Tweet
5 Reasons to Develop AI Systems In-House
1: The Best Core Technologies Are Open-source Anyway
The academic origins of open-source GPU-accelerated machine learning frameworks and libraries over the last ten years have made it all but impossible for well-funded tech giants to cloister promising new AI technologies into patent-locked, proprietary systems.
This is partly because nearly all the seminal contributing work has been the result of international collaborations involving some mix of academic research bodies and government or commercial institutions, and because of the permissive licensing2 that facilitated this level of global cooperation.
With occasional exceptions for the military sector3 and parts of Asia4, state-funded research is publicly accountable by necessity, while commercial attempts to take promising code into private branches would starve them, fatally, of ongoing community insight and development.
Ultimately all the major tech players were forced to join the open-source AI ecostructure5 in the hope that some other differentiating factor, such as Microsoft's business market capture, Amazon's gargantuan consumer reach, or Google's growing data mountains could later reap unique corporate benefits.
This unprecedented level of transparency and open technology gifts any private commercial project with free world-class machine learning libraries and frameworks, all not only adopted and well-funded (though not owned) by major tech players, but also proofed against subsequent revisionist licensing.
2: Protecting Corporate IP
Most in-house AI projects have a more fragile angle on success than the FAANG companies, such as a patentable use-case concept or the leveraging of internal consumer data — instances where the AI stack configuration and development is a mere deployment consideration rather than a value proposition in itself.
In order to avoid encroachment, it may be necessary to tokenize transactions6 that take place through cloud infrastructure, but keep local control of the central transaction engine.
Where client-side latency is a concern, one can also deploy opaque but functional algorithms derived from machine learning methods, rather than trusting the entirety of the system to the cloud, and encrypt or tokenize data returns for local analysis.
Such hybrid approaches have become increasingly common7 in the face of growing breach reports8 and hacking scandals over the last twenty years9.
3: Keeping Control of Data Governance and Compliance
The specificity of the input data for machine learning models is so lost in the training process that concerns around governance and management of the source training data might seem irrelevant, and shortcuts tempting.
However, controversial algorithm output10 can result in a clear inference of bias, and in embarrassingly public audits11 of the unprocessed training source data and the methodologies used.
In-house systems are more easily able to contain such anomalies once identified. This approach ensures that any such roadblocks in machine learning development neither overstep the terms and conditions of the cloud AI providers nor risk infringing the lattice of varying location-specific privacy and governance legislation that must be considered when deploying cloud-based AI processing systems.
4: AIaaS Can Be Used for Rapid Prototyping
The tension between in-house enterprise AI and cloud-based or outsourced AI development is not a zero-sum game. The diffusion of open-source libraries and frameworks into the most popular high-volume cloud AI solutions12 enables rapid prototyping and experimentation, using core technologies that can be moved in-house after the proof-of-concept is established, but which are rather more difficult for a local team to investigate creatively on an ad-hoc basis.
Rob Thomas, General Manager of IBM Data and Watson AI, has emphasized the importance of using at-scale turnkey solutions to explore various conceptual possibilities for local or hybrid AI implementations, asserting that even a 50% failure rate will leave an in-house approach with multiple viable paths forward13.
Additionally, Nvidia's $40-billion acquisition of Arm promises an industry shift towards more and better local ML resources in the years ahead14, with an emphasis on distributed and edge resources that are not managed by central cloud providers, except for the exigencies of bandwidth and low latency.
5: High-Volume Providers Are Not Outfitted for Marginal Use Cases
If an in-house project does not center around the highest-volume use cases of external providers, such as computer vision or natural language processing, deployment and tooling is likely to be more complicated and time-consuming. It’s also likely to be lacking in quick-start features such as applicable pre-trained models, suitably customizable analytics interfaces, or apposite data pre-processing pipelines.
Not all marginal use cases of this nature are SMB-sized. They also occur in industries and sectors that may be essential but operate at too limited a scale or within such high levels of oversight (such as the nuclear and financial industries) that no 'templated' AI outsourcing solution is ever likely to offer adequate regulatory compliance frameworks across territories, or enough economy of scale to justify investment on the part of off-the-shelf cloud AI providers.
Commodity cloud APIs can also prove more expensive and less responsive in cases where the value of a data transaction lies in its scarcity and exclusivity rather than its capacity to scale at volume or address a large captive user base at a very low latency.
with Iflexion’s data engineers.
5 Reasons to Adopt AI via External Provisioning
1: Development, Staging, and Deployment Are Accelerated
It's been noted15 that some of the most prominent tech companies can take an extraordinarily long time to formulate, test, develop, and deploy their in-house machine learning systems:
- Uber's machine learning platform Michelangelo took five years of ongoing development16.
- Airbnb's framework-agnostic machine learning platform Bighead was nearly four years in the making17.
- Netflix's machine learning program covers almost the last ten years of GPU-driven AI, and has burned through millions of dollars in research, much of it wasted18.
Smaller companies may not have the initial financial buoyancy to support such long lead times towards launching even a medium-scale machine learning deployment. The kind of rapid prototyping referred to in #4 above is costly to replicate locally, particularly for systems that are ultimately intended to pass through multiple stages of authentication and testing on high-scale international networks.
Cloud-based compute instances and VMs are cheap to spin up and discard at both the PoC stage and later phases of development and deployment. It's easier to prove viability in the target environment for a machine learning application or framework, rather than burden an in-house project with a painful migration stage that may have a profound effect on tooling, performance, and even core assumptions around the initiative.
In reference to point #5 above, many startups will seed their platform on templated, high-volume platforms with only short-term commitment in order to demonstrate that the product is viable, and later migrate to a cheaper hybrid or in-house solution once the computing costs, architecture, and central business model are established.
2: Better Talent Availability
The industrial scale of the larger cloud ML providers is accompanied by a high level of available and experienced quality developer and data scientist talent. In this area too, it can be cheaper and quicker to accommodate your project to the blunter edges of 'commoditized' provisioning than to develop in-house talent, either by attempting to train up existing hires or by seeking new suitable candidates.
Candidate availability has changed under the tumultuous global events of 2020. While the destabilizing effects of the lockdown have released a great deal of developer talent back to the market as, we analyzed in our latest piece of the future of data science in the age of COVID-19, any smaller company is likewise prone to the disastrous conditions that flooded the market in this way in the first place.
3: Open Source and Hybrid Adoption Makes Vendor Lock-In Harder to Achieve
The popularity of hybrid adoption and the wide-spread diffusion of open-source machine learning libraries and frameworks in outsourced provisioning makes vendor lock-in more difficult to achieve than with pure cloud provisioning19, or in other commercial vendor models where proprietary systems can obstruct interoperability and portability.
Despite this, the ML outsourcing sector is able to remain competitive because high-intensity, low-latency computing power is essential in many production machine learning models, and budgeting for on-demand access to industrialized compute instances will almost always be cheaper than attempting to replicate similar resources at a private localized scale.
A key factor in this choice is the volume and continuity of model training that a production system will ultimately need. Where a model can be developed locally and updated incrementally on the cloud, renting a GPU (or TPU, or other AI-centric ASIC hardware) may be far more economical than paying to maintain local compute capacity, in terms of acquisition, maintenance and running costs.
4: Less Risk of Crippling Technical Debt
For on-premises machine learning solutions, the likeliest form of 'lock-in' occurs when an already-brittle workflow needs to adapt to an essential change in one of its core components. This usually involves a version upgrade (or even abandonment) of an established open-source AI library or framework, but can also center around adapting a system to new and better hardware.
Though there's no explicit financial motive involved here, technical debt of this nature can prove expensive enough to threaten a local machine learning initiative, or else leave it stuck in a developmental cul-de-sac.
Examples include the advent of TensorFlow 2.0, which integrated Keras as a low-level function whilst forcing sector-wide adaptation for tooling and workflows; the death of Theano in 201720; and the ongoing stress of maintaining multiple concurrent installations of Nvidia's cuDNN drivers for its CUDA GPU libraries21, due to delays in updates for critical libraries in a project's workflow.
In 2015, Google published a report into the endemic nature of technical debt in machine learning systems and found a number of system-level 'anti-patterns' that are prone to undermine their longevity and maintainability, not least of which is coping with changes in the structure or nature of contributing data, as well as adapting existing workflows to later versions of core modules. In 2020, renowned ML-engineer Matthew McAteer who also worked for Google revisited the 2015 report.25 In short, McAteer points out that while many solutions that can tackle ML technical debt have surfaced since then, the issue still persists.
Migration and upgrade scenarios for local machine learning frameworks are extremely difficult to contend with when relying only on local talent and sporadic community help for open-source code repositories. Engaging consultants in such situations can eradicate any cost-savings achieved in local vs cloud-based AI deployment.
Despite preview access to new software and hardware innovations, high-volume providers may offer upgrade paths and migration solutions later than an in-house team of AI developers can; but when the solution does arrive, the sheer commercial concentration of effort means that an external provisioning solution is more likely to be resilient and flexible, enabling smoother transition paths at both the software and hardware level.
5: At Least Part of Your Solution Will Be Cloud-Based Anyway
Gartner predicts that worldwide public cloud spending will reach nearly $500 billion in 2022.23 — sectors now synonymous24 with machine learning and AI-driven forecasting and reporting.
The post-outbreak economy is expected to center on analytics as the driving force of the ongoing AI revolution, in a context of frozen budgets and precarious business forecasts under COVID-19.
Statistically, your machine learning project is most likely to also center on some sub-sector of ML-driven analytics, such as computer vision, NLP, or time-series forecasting — areas where existing cloud providers can offer immediate, flexible and scalable solutions with limited long-term commitment.
Tweet
Sources (25)from our expert team.
WANT TO START A PROJECT?
It’s simple!