Kinara Edge AI Processor Tackles the Monstrous Compute Demands of Generative AI and Transformer-Based Models
December 20, 2023 | BUSINESS WIREEstimated reading time: 2 minutes
Kinara, Inc., launched the Kinara Ara-2 Edge AI processor, powering edge servers and laptops with high performance, cost effective, and energy efficient inference to run applications such as video analytics, Large Language Models (LLMs), and other Generative AI models. The Ara-2 is also ideal for edge applications running traditional AI models and state-of-the-art AI models with transformer-based architectures. With an experientially enhanced feature set and more than 5-8 times the performance of its first-generation Ara-1 processor, Kinara’s Ara-2 combines real-time responsiveness with high throughput, merging its proven latency optimized design with perfectly balanced on-chip memories and high off-chip bandwidth to execute very large models with extremely low latency.
LLMs and Generative AI in general have become incredibly popular, but most of the associated applications are running on GPUs in data centers and are burdened with high latency, high cost, and questionable privacy. To overcome these limitations and put the compute literally in the hands of the user, Kinara’s Ara-2 simplifies the transition to the edge with its support for the 10’s of billions of parameters used by these Generative AI models. Furthermore, to seamlessly facilitate the migration from expensive GPUs for a wide variety of AI models, the compute engines in Ara-2 and the associated software development kit (SDK), are specifically designed to support high-accuracy quantization, a dynamically moderated host runtime, and direct FP32 support.
“With Ara-2 added to our family of processors, we can better provide customers with performance and cost options to meet their requirements. For example, Ara-1 is the right solution for smart cameras as well as edge AI appliances with 2-8 video streams, whereas Ara-2 is strongly suited for handling 16-32+ video streams fed into edge servers, as well as laptops, and even high-end cameras,” said Ravi Annavajjhala, Kinara’s CEO. “The Ara-2 enables better object detection, recognition, and tracking by using its advanced compute engines to process higher resolution images more quickly and with significantly higher accuracy. And as an example of its capabilities for processing Generative AI models, Ara-2 can hit 10 seconds per image for Stable Diffusion and tens of tokens/sec for LLaMA-7B."
In October, Ampere welcomed Kinara into the AI Platform Alliance with the primary goal of reducing system complexity and promoting better collaboration and openness with AI solutions and ultimately delivering better total performance and increased power and cost efficiency than GPUs. Ampere’s Chief Evangelist Sean Varley said, “The performance and feature set of Kinara’s Ara-2 is a step in the right direction to help us bring better AI alternatives to the industry than the GPU-based status quo.”
The Ara-2 also offers secure boot, encrypted memory access, and a secure host interface to enable enterprise AI deployments with even greater security. Kinara also supports Ara-2 with a comprehensive SDK that includes a model compiler and compute-unit scheduler, flexible quantization options that include the integrated Kinara quantizer as well as support for pre-quantized PyTorch and TFLite models, a load balancer for multi-chip systems, and a dynamically moderated host runtime.
Ara-2 is available as a stand-alone device, a USB module, an M.2 module, and a PCIe card featuring multiple Ara-2’s.
Suggested Items
ispace-EUROPE, the Italian Space Agency (ASI) Sign Payload Services Agreement
12/30/2024 | BUSINESS WIREispace EUROPE S.A., the Luxembourg-based subsidiary of ispace, inc., and the Italian Space Agency (ASI) have signed a payload services agreement to transport a Laser Retroreflector Array (LaRA2) to enable accurate position measurements on the Moon via laser ranging experiments, the two organizations announced today.
Beyond Design: AI-driven Inverse Stackup Optimization
12/26/2024 | Barry Olney -- Column: Beyond DesignArtificial intelligence (AI) is transforming how we conceptualize and design everything from satellites to PCBs. Traditionally, stackup planning is a manual process that can be multifaceted and relies heavily on the designer's expertise. Despite having best practices and various field solvers to optimize parameters, stackup planning remains challenging for complex designs with advanced packaging, several layers, multiple power pours, and controlled impedance requirements.
MicroCare Welcomes Dagmara Charyk as Regulatory Affairs Manager
12/13/2024 | MicroCare Corp.MicroCare, a leading provider of critical cleaning solutions, is excited to announce the appointment of Dagmara Charyk as the new Regulatory Affairs Manager. In this key role, Dagmara will oversee regulatory compliance, product registrations, and industry standards, further strengthening the MicroCare commitment to delivering safe and effective products to its customers.
Nano Dimension’s Additive Electronics Product Line - Essemtec - Received a Global Technology Award
12/11/2024 | Nano DimensionNano Dimension Ltd, a leading supplier of Additively Manufactured Electronics (AME) and multi-dimensional polymer, metal & ceramic Additive Manufacturing (AM) 3D printing solutions, announced that its Tarantula Underfill system from its Essemtec product lines received the Global Technology Award for Innovation at the SMTA International 2024 Show in Chicago, Illinois.
Has the Time Finally Come for Tin-nickel Plating?
12/09/2024 | Marcy LaRont, PCB007 MagazineIn the 1980s, Electrochemicals, Inc. (now Electrochemical Products) made a significant shift from furniture and industrial goods to electronics and manufacturing. During this journey, a tin-nickel plating alloy was developed. In 1984, Mike Carano, a young engineer, published a paper on tin-nickel plating alloy, but after some initial attention, the plating solution fell into obscurity. Today, PCB fabrication looks largely the same, yet changes are afoot, chiefly due to the demands for very fine feature capability on printed circuit boards, as well as environmental sustainability. So, in this fascinating conversation with Mike Carano and Happy Holden, we take another look at tin-nickel and its advantages in both performance and sustainability.