AI at the edge: simplifying infrastructure with Cisco and Canonical

Legacy infrastructure was not designed for the requirements of the AI era. While large-scale model training remains centralized in data centers, test-time inference is rapidly shifting to the edge to reduce latency and bandwidth consumption. This shift creates a new frontier for enterprise AI, but deploying at the edge introduces significant manual complexity, interoperability issues, and security vulnerabilities.

To address these challenges, Cisco and Canonical have developed a new Cisco Validated Design (CVD). This guide details how to leverage the Canonical portfolio on the Cisco Unified Edge system to deliver scalable, secure, and cost-efficient AI-ready infrastructure. In this article, we’ll whet your appetite by highlighting the key challenges, technologies, and solutions explored in the guide.

The challenges of legacy infrastructure for AI

Enterprises attempting to deploy AI use cases on traditional edge infrastructure typically face five critical bottlenecks:

• Hardware Limitations: Lack of specialized acceleration (GPUs) and high-density compute in small form factors.
• Architectural Rigidity: Static environments that cannot easily pivot between virtualized and containerized workloads.
• Scaling Inefficiency: Difficulty in managing thousands of geographically dispersed sites consistently.
• Cost Prohibitions: High CapEx for “rip-and-replace” cycles and high OpEx for manual site maintenance.
• Software Fragmentation: Version lag, lack of security patching (CVEs), and vendor lock-in.

Let’s dig into how our joint solution with Cisco addresses these challenges.

The software layer: A unified open source stack

The solution begins with a hardened software foundation provided by Canonical. By decoupling the application layer from the underlying hardware, enterprises can modernize legacy operations without manual rebuilds.

• Ubuntu Pro: Provides a stable, 15-year security maintenance lifecycle, backporting of critical fixes, and seamless public cloud integration.
• Data Science Stack (DSS): A ready-to-use environment for data scientists to develop and deploy models without worrying about underlying library dependencies.
• Charmed Operators: Automated operations for popular AI/ML toolkits (e.g., Kubeflow, MLflow), enabling consistent deployment and “Day 2” operations across the fleet.

The hardware layer: Converged infrastructure for AI

AI at the edge requires hardware that is both rugged and high-performing. The Cisco Unified Edge is a purpose-built system that converges compute, networking, storage, and security into a compact footprint.

Hardware certification

A key advantage of this partnership is the Canonical certification program. The Cisco UCS hardware is Ubuntu Certified, meaning Canonical works directly with Cisco to ensure the OS kernel is optimized for this specific platform. Running on this hardware, Ubuntu Server 24.04.3 LTS provides a stable, trusted open source foundation for edge applications.

The design guide we’ve developed with Cisco utilizes the Cisco UCS XE9305 chassis, which provides a variety of features to support inference at the edge:

• Form Factor: A 3-Rack-Unit (3RU) short-depth platform designed for space-constrained edge environments.
• Compute Nodes: Hosts up to five Cisco UCS XE130c nodes.
• Processing: 6th Gen Intel Xeon SoC processors (up to 32 cores per node).
• Memory: Up to 768GB DDR5 for high-throughput data processing.
• Acceleration: Dedicated NVIDIA L4 Tensor Core GPUs, providing energy-efficient AI inference.

Deployment flexibility: From VMs to Kubernetes

Edge environments often require a mix of legacy and modern workloads. The Cisco and Canonical solution supports multiple deployment models on a single platform, solving the architectural rigidity challenge:

• System containers and VMs with LXD: LXD treats containers like lightweight virtual machines, offering a VM-like experience with the efficiency of containers. It is ideal for hosting full Linux distributions or infrastructure services with minimal overhead.
• Canonical Kubernetes: For orchestrated, cloud-native applications, Canonical Kubernetes delivers an enterprise-grade distribution that is fully upstream-aligned.
• Canonical MicroCloud: This lightweight, automated private cloud solution is purpose-built for resource-constrained environments. It combines LXD, MicroCeph for storage, and MicroOVN for networking into a self-managing stack.

Zero-Touch operations and security

Managing thousands of edge locations is an operational bottleneck. This solution utilizes Cisco Intersight, a cloud-based management platform, to enable Zero-Touch Provisioning (ZTP).

By using curated Blueprints, administrators can automate the deployment of the entire stack, from hardware firmware to the OS and Kubernetes layers. This eliminates manual configuration errors and ensures site-to-site consistency.

This foundation is reinforced by multi-layered protection, utilizing Ubuntu Pro for continuous CVE patching and MicroOVN to ensure network isolation for sensitive AI models and data.

Conclusion

The shift to edge AI demands a departure from traditional infrastructure silos. By combining Cisco’s modular, high-performance hardware with Canonical’s automated open source software stack, enterprises can build a future-ready platform that scales without the need for constant “rip-and-replace” cycles.

Would you like to explore the full technical specifications and deployment steps?

Read the full Cisco Unified Edge and Canonical Design Guide

If you have any questions, you can contact us directly:

Enrico Panetta, Alliance Field Engineer – enrico.panetta@canonical.com

Further reading:

Ubuntu Server documentation 

Canonical MicroCloud documentation