Embedded & Edge AI Hardware: The Rubber Meets the Road for Production Scale

Inside this Report

The embedded and edge AI marketplace is rapidly evolving behind widespread advances in processing technology, software tooling, model availability, and open source initiatives. The market opportunity for edge AI spans the gamut of embedded hardware footprints from ultra-low-power sensors to high-performance near edge infrastructure, which all stand to benefit from the continuous innovation of core technology providers trying to center themselves within this new revenue stream. As with such fragmentation in target edge system types, footprints, deployment parameters, and vertical end users, no single company can completely take over the expansive edge AI computing market. It is under near-constant change with new solutions and players entering the market, including small upstarts as well as multi-billion-dollar organizations that are firmly entrenched in the embedded sector already.

Within this report is VDC's research-driven commentary about the largest influences and players in the burgeoning commercial market for embedded AI hardware sold to third-party/external OEMs/ODMs, system integrators, service providers, and enterprise/industrial end users. The market is increasingly broad with varying regional adoption and real-world deployments of edge AI solutions into domains such as aerospace and defense, automotive, communications and networking, consumer electronics, digital security and surveillance, energy and power, industrial automation and control, medical devices and healthcare, retail automation and digital signage, robotics, transportation and logistics, and other verticals. This research also includes an in-depth analysis of the competitive landscape for commercial embedded and edge AI hardware solutions of various form factors.

This market research report investigates the global merchant TAM for embedded and edge AI hardware, including dedicated sizing and trends analysis across the markets for discrete processors, computing modules, motherboards, single-board computers, and integrated systems and servers. It features a deep investigation into the key drivers for adoption, production scale up, and transformation in the embedded AI hardware space supported by highly granular market sizing, research interviews and ongoing conversations/projects with several dozen key ecosystem members and leading device suppliers, as well as interviews with and large-scale surveys of decision makers at product development organizations.

What Questions are Addressed?

• How large is the global market for embedded and edge AI hardware, and what is driving its growth?
• Which vertical markets are leading the charge for full-scale production rollouts of edge AI?
• What regional factors and regulations will impact the development of edge AI solutions?
• Who are the leading providers of embedded and edge AI processors, modules, boards, SBCs, and integrated systems/servers and the hardware acceleration within their offerings?
• Which types of AI hardware accelerators are expected to grow the fastest for different form factors?
• Which embedded stack components and capabilities/features have the most synergy with edge AI in engineering projects today?
• How much edge AI performance is needed in today's edge systems and three years from now?

Who Should Read this Report?

This report was written for those making critical decisions regarding product, market, channel, and competitive strategy and tactics. This report is intended for senior decision-makers who are developing, or are a part of the ecosystem of, IoT, embedded, and/or edge AI computing solutions, including:

Technology Providers in this Report:

Demand-side Research Overview

VDC launches numerous surveys of the IoT and embedded engineering ecosystem every year using an online survey platform. To support this research, VDC leverages its in-house panel of more than 30,000 individuals from various roles and industries across the world. Our global Voice of the Engineer survey recently captured insights from a total of 600 qualified respondents. This survey was used to inform our insight into key trends, preferences, and predictions within the engineering community.

Executive Summary

Edge AI is sweeping the embedded hardware market faster and with more impact than any individual technology revolution before - whether multicore processing, hardware-based security, M2M communications, IoT connectivity platforms, or local intelligent analytics. The edge AI revolution has transformed decision-making in product development and reshaped the very core of hardware architecture choice and design. It is a blanket trend that is driving rapid advances in many established and new embedded/OEM use cases, both to internal systems and services optimization as well as for external/customer applications. Edge AI is creating a rare inflection point in the embedded sector, driving transformation in a market long characterized by slow adoption and resistance to major architectural change due to entrenched design dependencies, stringent cost constraints, regulatory compliance demands, security requirements, and other barriers.

Mounting data volumes in the field, despite great advances in connectivity and embedded storage technologies through the past several years, continues to challenge data utilization strategies. The only way to achieve maximum autonomy and extract the maximum value from far edge/endpoint and operational data is to employ edge AI at several levels of end-to-end IoT and industrial solutions. However, each edge AI deployment, even within the same industries or environments, has its own unique considerations for power availability, network access, connected devices (i.e., cameras, sensors) and infrastructure, data types, etc. As a result, a wide variety of AI hardware solutions and integrated or attached workload acceleration technologies are in high demand within the IoT, embedded, and edge computing market to cater to the vast needs of manufacturers, integrators, and end users.

Key Findings:

• The global embedded and edge AI hardware market is projected to grow from ~$17.8B in 2024 to over $46.4B by 2029, reflecting a CAGR of 21.1%.
• Integrated NPUs are the fastest-growing AI accelerator type throughout the forecast period for discrete embedded MPUs, MCUs, and SoCs.
• In the embedded computing modules market (CoMs and SoMs), integrated GPU-driven acceleration accounts for nearly half of edge AI revenue shipments, with adoption rates even higher when used as attached accelerators for motherboards, SBCs, and integrated systems/servers.
• The largest vertical markets for embedded and edge AI hardware are automotive, communications and networking, consumer electronics, industrial automation and control, and retail automation and digital signage, with other segments such as medical/healthcare and robotics rapidly emerging.
• The Americas holds the largest share of the embedded/OEM AI systems and servers market, while Asia-Pacific's manufacturing concentration makes it the leading region for component processors and computing modules.
• VDC Research's Voice of the Engineer survey data shows a strong correlation between embedded AI projects/products and features such as mobile or battery-powered form factors, security enhancements (hardware and software), remote application deployment and management, and machine vision/object detection.

Report Excerpt

The embedded and edge AI hardware market is expanding rapidly, increasing its share within the broader markets for embedded processors, computing modules, motherboards, single-board computers, and integrated systems/servers. Through its implementation in both general-purpose hardware as well as via the growing portfolios of dedicated product families, embedded and edge AI capabilities are becoming table stakes for engineering organizations and end users worldwide and across industries. AI and machine learning technologies are unlocking significant value in the field, enabling the next data revolution with advancements in vision and imaging systems, sensor platforms, power and connectivity infrastructure, user interaction, and other domains.

Although AI and machine learning are not new to the embedded market, the surrounding ecosystem of hardware accelerators, software development platforms, open-source models, and AI frameworks has radically expanded their potential, from MCU-based sensors to many-core systems and servers. At the same time, the market opportunity for embedded and edge AI isn't confined to only greenfield deployments or new infrastructure rollouts. Given the long deployment lifetimes typical in many of the leading embedded hardware markets, there is strong demand for computing solutions to augment existing deployments or designs with attached or in-line edge AI capabilities - transforming "dumb" sensors, cameras, client devices, machinery, and other operational systems into intelligent platforms to achieve new efficiencies, controls, or levels of automation.

1 Note: Exhibit has been modified for the Executive Brief. Granular market forecast data is available within the report.

Scope & Methodology

AI, Machine Learning & Neural Networks

For the purposes of this report, VDC defines artificial intelligence as the ability of machines to mimic the cognitive functions of humans and carry out complex tasks in an autonomous or semi-autonomous fashion.

Machine learning (ML) is an application of AI that allows a system to improve its operation through the analysis and reimplementation of collected data without the need for explicit reprogramming. Production ML systems generally consist of a large bundle of code and tools surrounding the core ML decision-making code. Data collection, feature extraction, process management tools, data verification, machine resource management, configuration, analysis tools, monitoring, and server infrastructure are all vital components for production ML systems. The objective of ML is to build a tool and compute-based framework that allows machines to combine inputs to produce useful predictions on never-before-seen data. Machine learning models can be static or dynamic. A static model is trained offline, exactly once, then run on this system for a period of time before being updated. A dynamic model is trained on an ongoing basis, with data continuously entering the system and training the model as it operates.

A neural network (NN) further optimizes an ML system by adding layers of nodes to a model. A node consists of one or more weighted input connections, a transfer function that combines the inputs, and an output connection to the larger model. These layers of nodes transform the data in useful ways before it is passed to the final output node of the ML algorithm to use in its decision or classification. NNs allow for much more complex actions to be performed on incoming data and are useful for nonlinear systems where an ML algorithm must distinguish between many different possibilities (e.g., "Is this image an apple, a bear, an egg, or a duck?") rather than binary yes or no decisions. NNs are classified as deep learning methods or algorithms that attempt to learn broader data representations and patterns, as opposed to inflexible, task-specific algorithms.

Methodology

VDC produces its forecasts from a combination of primary and secondary data, including its own surveys and in- depth interviews with OEMs and industry executives; internal historical data sets and market models; public and privately disclosed numbers from vendors and customers, industry organizations, trade publications, and conferences; and our analysts' extensive base of industry experience and knowledge.

In this report, we size the market for merchant embedded and edge hardware supporting AI applications that are built for, and deployed on, OEM and embedded systems. It does not include the captive portion of the market, which includes hardware designed and built internally at OEMs solely for internal use or integration into their own larger systems or end products.

The embedded and edge AI hardware market sizing estimates and forecasts are based on a combination of:

• (1) VDC's longstanding expertise and ongoing coverage of the total market opportunity for merchant processors, boards, modules, and integrated systems;
• (2) frequently updated engineering survey data from product development organizations designing, building, deploying, and managing edge AI solutions;
• (3) in-depth interviews with decision-makers at OEMs and other embedded technology customers; and
• (4) regular discussions with embedded and edge technology providers and broader ecosystem participants.