本文介绍rapidlayout库的onnxruntime gpu推理示例。该库用于文档图像版面分析，集成多种模型。在gpu环境下，onnxruntime-gpu推理速度比cpu快10倍。文中给出安装步骤，分别测试了cpu和gpu版onnxruntime的推理耗时，gpu平均耗时0.0440s，远快于cpu的0.9425s，还说明关键是设置use_cuda=true参数。
☞☞☞AI 智能聊天, 问答助手, AI 智能搜索, 免费无限量使用 DeepSeek R1 模型☜☜☜
RapidLayout onnxruntime GPU推理示例
RapidLayout简介
RapidLayout Github: https://github.com/RapidAI/RapidLayout
在线demo: https://huggingface.co/spaces/SWHL/RapidLayout
RapidLayout库主要是做文档类图像的版面分析。具体来说，就是分析给定的文档类别图像（论文截图、研报等），定位其中类别和位置，如标题、段落、表格和图片等各个部分。该库集成了PP-Structure中3个版面分析模型、360开源的4个版面分析模型，大家可按需使用。
版面分析落地场景：一般用于结构化文档使用。将PDF中每一页转为图像，用版面分析检测出图像中各个元素，像段落、表格、公式，然后将各个元素送到各个专用模型下识别，最后汇总做版面恢复为Word或者TXT格式。
GPU推理速度结论：onnxruntime-gpu推理速度有10倍提升。原因是版面分析输入图像尺寸固定。
前提条件
启动该项目时，需要选择具有GPU的运行环境（V100），可选下图红框任意一个：
对比结果
| 推理 | 推理速度(s/img) |
|---|---|
| CPU | 0.9425 |
| GPU V100 | 0.0440 |
安装rapid_layout
⚠️注意：默认安装cpu版的onnxruntime库
In [1]
!pip install rapid_layout
Looking in indexes: https://mirror.baidu.com/pypi/simple/, https://mirrors.aliyun.com/pypi/simple/ Collecting rapid_layout Downloading https://mirrors.aliyun.com/pypi/packages/81/49/2aecd89bb7a91d745565257a580b2f7869dcf722a9cad03c106f1c1b0a31/rapid_layout-0.2.0-py3-none-any.whl (6.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.8/6.8 MB 636.5 kB/s eta 0:00:0000:0100:01 Requirement already satisfied: Pillow in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rapid_layout) (10.3.0) Requirement already satisfied: numpy<2,>=1.21.6 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rapid_layout) (1.26.4) Requirement already satisfied: onnxruntime>=1.7.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rapid_layout) (1.18.0) Requirement already satisfied: opencv-python>=4.5.1.48 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rapid_layout) (4.10.0.84) Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rapid_layout) (2.32.3) Requirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from rapid_layout) (4.66.4) Requirement already satisfied: coloredlogs in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapid_layout) (15.0.1) Requirement already satisfied: flatbuffers in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapid_layout) (24.3.25) Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapid_layout) (24.1) Requirement already satisfied: protobuf in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapid_layout) (3.20.3) Requirement already satisfied: sympy in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime>=1.7.0->rapid_layout) (1.12.1) Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from requests->rapid_layout) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from requests->rapid_layout) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from requests->rapid_layout) (2.2.2) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from requests->rapid_layout) (2024.6.2) Requirement already satisfied: humanfriendly>=9.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from coloredlogs->onnxruntime>=1.7.0->rapid_layout) (10.0) Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from sympy->onnxruntime>=1.7.0->rapid_layout) (1.3.0) Installing collected packages: rapid_layout Successfully installed rapid_layout-0.2.0
测试用CPU版onnxruntime推理耗时
In [2]
import cv2from rapid_layout import RapidLayoutfrom pathlib import Path
layout_engine = RapidLayout(conf_thres=0.5, model_type="pp_layout_cdla")# warm uplayout_engine("images/12027_5.png")
elapses = []
img_list = list(Path('images').iterdir())for img_path in img_list:
boxes, scores, class_names, elapse = layout_engine(img_path) print(f"{img_path}: {elapse}s")
elapses.append(elapse)
avg_elapse = sum(elapses) / len(elapses)print(f'avg elapse: {avg_elapse:.4f}')2024-06-25 21:11:44,758 - DownloadModel - INFO: /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/rapid_layout/models/layout_cdla.onnx already exists 2024-06-25 21:11:45,088 - rapid_layout - INFO: pp_layout_cdla contains ['text', 'title', 'figure', 'figure_caption', 'table', 'table_caption', 'header', 'footer', 'reference', 'equation']
images/1712652165000_6.png: 0.8755233287811279s images/12027_5.png: 0.9934070110321045s images/基于子字单元的神经机器翻译未登录词翻译分析_5.png: 0.894378662109375s images/000968_蓝焰控股2023年三季度报告_8.png: 0.912381649017334s images/20020304_3.png: 0.9924137592315674s images/layout.png: 0.9869406223297119s avg elapse: 0.9425
安装GPU版onnxruntime
In [1]
!pip uninstall onnxruntime -y !pip install onnxruntime-gpu
Found existing installation: onnxruntime 1.18.0 Uninstalling onnxruntime-1.18.0: Successfully uninstalled onnxruntime-1.18.0 Looking in indexes: https://mirror.baidu.com/pypi/simple/, https://mirrors.aliyun.com/pypi/simple/ Collecting onnxruntime-gpu Downloading https://mirrors.aliyun.com/pypi/packages/67/ca/f9c9c338eb1075e53d785cf1a62a50388f2d210d38fc8f9dafd7d145039a/onnxruntime_gpu-1.18.0-cp310-cp310-manylinux_2_28_x86_64.whl (199.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.8/199.8 MB 552.3 kB/s eta 0:00:0000:0100:10 Requirement already satisfied: coloredlogs in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime-gpu) (15.0.1) Requirement already satisfied: flatbuffers in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime-gpu) (24.3.25) Requirement already satisfied: numpy>=1.21.6 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime-gpu) (1.26.4) Requirement already satisfied: packaging in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime-gpu) (24.1) Requirement already satisfied: protobuf in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime-gpu) (3.20.3) Requirement already satisfied: sympy in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from onnxruntime-gpu) (1.12.1) Requirement already satisfied: humanfriendly>=9.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from coloredlogs->onnxruntime-gpu) (10.0) Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages (from sympy->onnxruntime-gpu) (1.3.0) Installing collected packages: onnxruntime-gpu Successfully installed onnxruntime-gpu-1.18.0
测试用GPU版onnxruntime推理耗时
In [3]
import cv2from rapid_layout import RapidLayoutfrom pathlib import Path# 注意：这里需要使用use_cuda指定参数layout_engine = RapidLayout(conf_thres=0.5, model_type="pp_layout_cdla", use_cuda=True)# warm uplayout_engine("images/12027_5.png")
elapses = []
img_list = list(Path('images').iterdir())for img_path in img_list:
boxes, scores, class_names, elapse = layout_engine(img_path) print(f"{img_path}: {elapse}s")
elapses.append(elapse)
avg_elapse = sum(elapses) / len(elapses)print(f'avg elapse: {avg_elapse:.4f}')2024-06-25 21:05:30,799 - DownloadModel - INFO: /opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/rapid_layout/models/layout_cdla.onnx already exists 2024-06-25 21:05:31,067 - rapid_layout - INFO: pp_layout_cdla contains ['text', 'title', 'figure', 'figure_caption', 'table', 'table_caption', 'header', 'footer', 'reference', 'equation']
images/1712652165000_6.png: 0.041887760162353516s images/12027_5.png: 0.042180538177490234s images/基于子字单元的神经机器翻译未登录词翻译分析_5.png: 0.0420379638671875s images/000968_蓝焰控股2023年三季度报告_8.png: 0.054517507553100586s images/20020304_3.png: 0.04352569580078125s images/layout.png: 0.0400235652923584s avg elapse: 0.0440
代码解释
In [ ]
<br/>
