Client

The customer is a RISC-V based AI accelerator company.

Challenge

The customer’s Accelerator hardware had support only for a minimal set of models through their NN software ecosystem. The goal of the project was to extend support for various other models.

MulticoreWare ran an end-to-end inference pipeline for various CNN and NLP Models for their architectures using their custom APIs. We had to write model demos for various CNN and NLP Models for their architectures and decompose unsupported TVM ops with supported ops in the customer’s own AI/ML compiler stack.

Solution

With our expertise on end-to-end model inference pipeline on various customer’s hardware in the past and present, our team of solution architects were able to add support for models such as ShuffleNet, YoloV5, HardNet, DDRNet, DLA and various other CNN, NLP and transformer based models and its variants on different architectures of the customer hardware. We also handled and supported different failures & bugs while supporting above mentioned models.

The correctness of the model inference pipeline was tested with PyTorch & ONNX reference code and using the metric PCC. Amidst the rapid development of APIs and features, we also adapted to the APIs and uplifted some of the models by comprehending memory layouts and configurations with minimal documentation.

Despite facing challenges with limited documentation and rapidly evolving repository development, our team successfully adapted to the new APIs, ensuring we met the customer’s requirements. This achievement highlights our ability to quickly learn and apply new technologies, overcoming obstacles to deliver quality results.