What’s Your Route to Enterprise AI Adoption?

Take a look at the current indicators that favor an in-house or platform-based approach to enterprise AI development, with an eye to emerging trends.

Published: February 10, 2022

Updated: July 01, 2022

By Martin Anderson

5 Reasons to Develop AI Systems In-House

1: The Best Core Technologies Are Open-source Anyway

The academic origins of open-source GPU-accelerated machine learning frameworks and libraries over the last ten years have made it all but impossible for well-funded tech giants to cloister promising new AI technologies into patent-locked, proprietary systems.

This is partly because nearly all the seminal contributing work has been the result of international collaborations involving some mix of academic research bodies and government or commercial institutions, and because of the permissive licensing² that facilitated this level of global cooperation.

With occasional exceptions for the military sector³ and parts of Asia⁴, state-funded research is publicly accountable by necessity, while commercial attempts to take promising code into private branches would starve them, fatally, of ongoing community insight and development.

Ultimately all the major tech players were forced to join the open-source AI ecostructure⁵ in the hope that some other differentiating factor, such as Microsoft's business market capture, Amazon's gargantuan consumer reach, or Google's growing data mountains could later reap unique corporate benefits.

This unprecedented level of transparency and open technology gifts any private commercial project with free world-class machine learning libraries and frameworks, all not only adopted and well-funded (though not owned) by major tech players, but also proofed against subsequent revisionist licensing.

2: Protecting Corporate IP

Most in-house AI projects have a more fragile angle on success than the FAANG companies, such as a patentable use-case concept or the leveraging of internal consumer data — instances where the AI stack configuration and development is a mere deployment consideration rather than a value proposition in itself.

In order to avoid encroachment, it may be necessary to tokenize transactions⁶ that take place through cloud infrastructure, but keep local control of the central transaction engine.

Where client-side latency is a concern, one can also deploy opaque but functional algorithms derived from machine learning methods, rather than trusting the entirety of the system to the cloud, and encrypt or tokenize data returns for local analysis.

Such hybrid approaches have become increasingly common⁷ in the face of growing breach reports⁸ and hacking scandals over the last twenty years⁹.

3: Keeping Control of Data Governance and Compliance

The specificity of the input data for machine learning models is so lost in the training process that concerns around governance and management of the source training data might seem irrelevant, and shortcuts tempting.

However, controversial algorithm output¹⁰ can result in a clear inference of bias, and in embarrassingly public audits¹¹ of the unprocessed training source data and the methodologies used.

In-house systems are more easily able to contain such anomalies once identified. This approach ensures that any such roadblocks in machine learning development neither overstep the terms and conditions of the cloud AI providers nor risk infringing the lattice of varying location-specific privacy and governance legislation that must be considered when deploying cloud-based AI processing systems.

4: AIaaS Can Be Used for Rapid Prototyping

The tension between in-house enterprise AI and cloud-based or outsourced AI development is not a zero-sum game. The diffusion of open-source libraries and frameworks into the most popular high-volume cloud AI solutions¹² enables rapid prototyping and experimentation, using core technologies that can be moved in-house after the proof-of-concept is established, but which are rather more difficult for a local team to investigate creatively on an ad-hoc basis.

Rob Thomas, General Manager of IBM Data and Watson AI, has emphasized the importance of using at-scale turnkey solutions to explore various conceptual possibilities for local or hybrid AI implementations, asserting that even a 50% failure rate will leave an in-house approach with multiple viable paths forward¹³.

Additionally, Nvidia's $40-billion acquisition of Arm promises an industry shift towards more and better local ML resources in the years ahead¹⁴, with an emphasis on distributed and edge resources that are not managed by central cloud providers, except for the exigencies of bandwidth and low latency.

5: High-Volume Providers Are Not Outfitted for Marginal Use Cases

If an in-house project does not center around the highest-volume use cases of external providers, such as computer vision or natural language processing, deployment and tooling is likely to be more complicated and time-consuming. It’s also likely to be lacking in quick-start features such as applicable pre-trained models, suitably customizable analytics interfaces, or apposite data pre-processing pipelines.

Not all marginal use cases of this nature are SMB-sized. They also occur in industries and sectors that may be essential but operate at too limited a scale or within such high levels of oversight (such as the nuclear and financial industries) that no 'templated' AI outsourcing solution is ever likely to offer adequate regulatory compliance frameworks across territories, or enough economy of scale to justify investment on the part of off-the-shelf cloud AI providers.

Commodity cloud APIs can also prove more expensive and less responsive in cases where the value of a data transaction lies in its scarcity and exclusivity rather than its capacity to scale at volume or address a large captive user base at a very low latency.

Explore your enterprise AI opportunities
with Iflexion’s data engineers.

Get in touch

5 Reasons to Adopt AI via External Provisioning

1: Development, Staging, and Deployment Are Accelerated

It's been noted¹⁵ that some of the most prominent tech companies can take an extraordinarily long time to formulate, test, develop, and deploy their in-house machine learning systems:

Uber's machine learning platform Michelangelo took five years of ongoing development¹⁶.
Airbnb's framework-agnostic machine learning platform Bighead was nearly four years in the making¹⁷.
Netflix's machine learning program covers almost the last ten years of GPU-driven AI, and has burned through millions of dollars in research, much of it wasted¹⁸.

Smaller companies may not have the initial financial buoyancy to support such long lead times towards launching even a medium-scale machine learning deployment. The kind of rapid prototyping referred to in #4 above is costly to replicate locally, particularly for systems that are ultimately intended to pass through multiple stages of authentication and testing on high-scale international networks.

Cloud-based compute instances and VMs are cheap to spin up and discard at both the PoC stage and later phases of development and deployment. It's easier to prove viability in the target environment for a machine learning application or framework, rather than burden an in-house project with a painful migration stage that may have a profound effect on tooling, performance, and even core assumptions around the initiative.

In reference to point #5 above, many startups will seed their platform on templated, high-volume platforms with only short-term commitment in order to demonstrate that the product is viable, and later migrate to a cheaper hybrid or in-house solution once the computing costs, architecture, and central business model are established.

2: Better Talent Availability

The industrial scale of the larger cloud ML providers is accompanied by a high level of available and experienced quality developer and data scientist talent. In this area too, it can be cheaper and quicker to accommodate your project to the blunter edges of 'commoditized' provisioning than to develop in-house talent, either by attempting to train up existing hires or by seeking new suitable candidates.

Candidate availability has changed under the tumultuous global events of 2020. While the destabilizing effects of the lockdown have released a great deal of developer talent back to the market as, we analyzed in our latest piece of the future of data science in the age of COVID-19, any smaller company is likewise prone to the disastrous conditions that flooded the market in this way in the first place.

3: Open Source and Hybrid Adoption Makes Vendor Lock-In Harder to Achieve

The popularity of hybrid adoption and the wide-spread diffusion of open-source machine learning libraries and frameworks in outsourced provisioning makes vendor lock-in more difficult to achieve than with pure cloud provisioning¹⁹, or in other commercial vendor models where proprietary systems can obstruct interoperability and portability.

Despite this, the ML outsourcing sector is able to remain competitive because high-intensity, low-latency computing power is essential in many production machine learning models, and budgeting for on-demand access to industrialized compute instances will almost always be cheaper than attempting to replicate similar resources at a private localized scale.

A key factor in this choice is the volume and continuity of model training that a production system will ultimately need. Where a model can be developed locally and updated incrementally on the cloud, renting a GPU (or TPU, or other AI-centric ASIC hardware) may be far more economical than paying to maintain local compute capacity, in terms of acquisition, maintenance and running costs.

4: Less Risk of Crippling Technical Debt

For on-premises machine learning solutions, the likeliest form of 'lock-in' occurs when an already-brittle workflow needs to adapt to an essential change in one of its core components. This usually involves a version upgrade (or even abandonment) of an established open-source AI library or framework, but can also center around adapting a system to new and better hardware.

Though there's no explicit financial motive involved here, technical debt of this nature can prove expensive enough to threaten a local machine learning initiative, or else leave it stuck in a developmental cul-de-sac.

Examples include the advent of TensorFlow 2.0, which integrated Keras as a low-level function whilst forcing sector-wide adaptation for tooling and workflows; the death of Theano in 2017²⁰; and the ongoing stress of maintaining multiple concurrent installations of Nvidia's cuDNN drivers for its CUDA GPU libraries²¹, due to delays in updates for critical libraries in a project's workflow.

In 2015, Google published a report into the endemic nature of technical debt in machine learning systems and found a number of system-level 'anti-patterns' that are prone to undermine their longevity and maintainability, not least of which is coping with changes in the structure or nature of contributing data, as well as adapting existing workflows to later versions of core modules. In 2020, renowned ML-engineer Matthew McAteer who also worked for Google revisited the 2015 report.²⁵ In short, McAteer points out that while many solutions that can tackle ML technical debt have surfaced since then, the issue still persists.

Migration and upgrade scenarios for local machine learning frameworks are extremely difficult to contend with when relying only on local talent and sporadic community help for open-source code repositories. Engaging consultants in such situations can eradicate any cost-savings achieved in local vs cloud-based AI deployment.

Despite preview access to new software and hardware innovations, high-volume providers may offer upgrade paths and migration solutions later than an in-house team of AI developers can; but when the solution does arrive, the sheer commercial concentration of effort means that an external provisioning solution is more likely to be resilient and flexible, enabling smoother transition paths at both the software and hardware level.

5: At Least Part of Your Solution Will Be Cloud-Based Anyway

Gartner predicts that worldwide public cloud spending will reach nearly $500 billion in 2022.²³ — sectors now synonymous²⁴ with machine learning and AI-driven forecasting and reporting.

The post-outbreak economy is expected to center on analytics as the driving force of the ongoing AI revolution, in a context of frozen budgets and precarious business forecasts under COVID-19.

Statistically, your machine learning project is most likely to also center on some sub-sector of ML-driven analytics, such as computer vision, NLP, or time-series forecasting — areas where existing cloud providers can offer immediate, flexible and scalable solutions with limited long-term commitment.

Discover 5 reasons to go for in-house AI system development — as well as 5 reasons to delegate it to external platform providers.

Conclusion

The logistics of data itself will determine where your project is best situated on the in-house vs cloud-based spectrum. Where the high value of internal company data is the driving force behind a new AI initiative, and where the AI framework serves only to facilitate this profitable scarcity, an in-house solution may be indicated — on the provision that leeway is left to move sections of the project to external provisioning in order to keep costs down, or to increase scale or performance as necessary.

Alternately, where data volume is high or requires a great deal of pre-processing (or is routinely crowdsourced), an in-house approach starts at a notable disadvantage and with a greater burden of speculative upfront investment.

Where your project's code is genuinely novel, and where its transformative potential is of greater value than the data that passes through it, a more cautious, hybrid approach with tokenization and/or encryption may be the best initial position, so long as both the business model and software architecture have made provision for subsequent migration in response to growing demand.

Finally, if your project centers chiefly on the most widely-diffused open-source libraries and use cases (such as NLP, computer vision, and analytics for recommender systems), it would be foolish not to turn to on-demand implementations from the high-volume providers that are increasingly specializing in these areas.