Milvus Community Conf 2020_澜起科技:基于异构硬件的 Milvus 及 Faiss 搜索加速

Embedding 搜索随着数据量和精度的提高,计算量和数据或索引文件的规模也越来越大,Embedding 算搜索中容量和计算性能容易成为整个系统的瓶颈。澜起科技有限公司尝试使用 FPGA 异构计算来加速大容量 Embedding 搜索的核心计算,并结合 Milvus 和 Faiss 开源软件,取得了很好的效果。

展开查看详情

1.Similarity Search Acceleration with HAS Hardware in Milvus/Faiss 澜起科技 Montage Technology 杨晔 Luke Yang (ye.yang@montage-tech.com)

2.Agenda  Montage Overview  Background of the Project  Montage HSA Acceleration for Milvus/Faiss  Performance Evaluation  Next Step Plan www.montage-tech.com Montage Technology Confidential and Proprietary | 2

3.Montage Company Profile • Date Founded: May 2004 • Employees: 300+ • IPO on the SSE STAR Market in July Xi’an Seoul, Korea 2019; Code: 688008 Shanghai Silicon Valley, USA Kunshan • Major Shareholders: Macao o China Electronics Investment o Intel Capital o Montage Employees o CITIC Securities • Products & Solutions: High- Headquarter and Worldwide Office Locations performance IC based solutions for cloud computing and artificial intelligent (AI) • Customers: Memory and server Shanghai (HQ) Kunshan Xi’an Macao Silicon Valley, Seoul, vendors and Internet users USA Korea worldwide www.montage-tech.com Montage Technology Confidential and Proprietary | 3

4.Montage Products & Solutions Jintide® CPU Memory Interface Chips Hybrid Security Memory Module (HSDIMM®) Data Center Provide silicon based solutions including Memory Interface Chip, Jintide® Server CPU and Hybrid Security Memory Module for cloud computing and artificial intelligence to enable Higher Data Rate & Safer Computing www.montage-tech.com Montage Technology Confidential and Proprietary | 4

5.Background of the Project  Similarity (Embedding) Search is widely adopted in Recommendation, Searching, Q&A, Robot… applications  Embedding Searching with Milvus/Faiss – Big dataset (10s Millions to Billions) – High precision required (95% to 99.99%) – Small delay required for online service  Bottlenecks – PCIE transferring bound of GPU – Memory capacity bound on GPU – Performance bound on CPU Application Architecture (https://milvus.io/) www.montage-tech.com Montage Technology Confidential and Proprietary | 5

6.Montage HSA Acceleration in Milvus/Faiss  Use HSA Device for Embedding Searching Queries – Heterogeneous System Architecture CPU DDR Bus Data/Index – Computing Engine + DDR memory PCIE PCIE – High efficiency computing engine for HSA embedding searching Computing GPU – Big memory capacity for big dataset/Index Engine DDR Bus  Big Data or Index stay in HSA Device Memory – For continuous queries Data/Index – Save the cost of data copy – High bandwidth access by HSA computing engine www.montage-tech.com Montage Technology Confidential and Proprietary | 6

7.Montage FPGA Acceleration as HSA Device in Milvus Software Framework Hardware Framework FPGA Implementation MAC LUT Sorter PCIE CPU SRAM DDR4 Memory * Red: Modified/Added code in Milvus v0.10.0 www.montage-tech.com Montage Technology Confidential and Proprietary | 7

8.Performance Evaluation - No Data Copy  Hardware – CPU: intel Skylake-16 cores 3GHz – Memory: 385GB – GPU: Nvidia Tesla P100 – 16GB mem – FPGA: Xilinx U250 (64GB DDR4)  Software – Ubuntu 18.04 Milvus: 0.10.0  Dataset – 100 Millions Random normalized vectors (64 Dimensions), dataset ~25.6GB, Index size ~7GB – Query batch size = 100;  Index – Faiss IVFPQ8 – Search Accuracy: >90% * GPU time does not include index copy time www.montage-tech.com Montage Technology Confidential and Proprietary | 8

9.Performance Evaluation - with Data Copy  Data/Index > GPU memory requirement  Multiple models or applications are sharing the GPU  Data/Index need to be updated while querying * GPU time includes index copy time www.montage-tech.com Montage Technology Confidential and Proprietary | 9

10.Next Step Plans  FPGA project improvements – Further Improve performance – Support Multi FPGA cards on single server  Share FPGA HAL (Hardware Abstraction Layer) code into Milvus – First v0.10.4 – v0.11 planned  Work with interested application vendor to further evaluation of Montage FPGA implementation – Contact ye.yang@montage-tech.com  Release FPGA implementation into Milvus (2021)  ASIC chip project of Montage www.montage-tech.com Montage Technology Confidential and Proprietary | 10

11.Questions? www.montage-tech.com Montage Technology Confidential and Proprietary | 11

12.Thanks! © 2019 Montage Technology. All rights reserved.