CN114821660A - Pedestrian detection inference method based on embedded equipment - Google Patents
Pedestrian detection inference method based on embedded equipment Download PDFInfo
- Publication number
- CN114821660A CN114821660A CN202210512803.XA CN202210512803A CN114821660A CN 114821660 A CN114821660 A CN 114821660A CN 202210512803 A CN202210512803 A CN 202210512803A CN 114821660 A CN114821660 A CN 114821660A
- Authority
- CN
- China
- Prior art keywords
- model
- pedestrian detection
- quantization
- quantizing
- calculating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 27
- 238000000034 method Methods 0.000 title claims abstract description 14
- 238000013139 quantization Methods 0.000 claims abstract description 41
- 230000004913 activation Effects 0.000 claims abstract description 19
- 238000011161 development Methods 0.000 claims abstract description 7
- 238000012549 training Methods 0.000 claims abstract description 7
- 238000004364 calculation method Methods 0.000 claims abstract description 4
- 238000012360 testing method Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 abstract description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides a pedestrian detection inference method based on embedded equipment, which is used for operating a pedestrian detection model with intensive calculation to low-power-consumption embedded equipment. The method is characterized in that edge end deep learning equipment based on a RISC-V framework is adopted, an MCU development board is used as a hardware platform, a Wujian100 open source IP is used as an MCU core, and an onboard serial port, an HDMI interface and an OV5640 camera are arranged. Acquiring training data, and training a pedestrian detection model MobileNet 1-SSD; calculating a quantization factor of the model weight; calculating to obtain an activation value quantization factor of each layer by minimizing mean square error; quantizing each operator in the model, quantizing the weight of the floating point type model into int8 data type, and quantizing the activation value into unt 8 data type; model reasoning and inverse quantization, wherein the reasoning result is inversely quantized into int32 data type; compiling and running the model on the MCU development board.
Description
Technical Field
The invention relates to a pedestrian detection inference method based on embedded equipment, and belongs to the technical field of pedestrian detection.
Background
In recent years, neural network models have been widely used in many fields and have achieved excellent results, especially in the field of pedestrian detection. However, the pedestrian detection neural network model has low inference efficiency and long inference time due to high model complexity and large model, and particularly operates in low-performance mobile equipment and low-power-consumption equipment. Therefore, how to design a model which has low resource consumption, can predict in real time and simultaneously ensure the prediction precision becomes a practical problem. On low-power consumption equipment similar to the MCU, a model with low resource consumption is needed, and in addition, a plurality of MCUs do not support floating point operation, so that the application of the model is limited. The model quantization has a good effect on solving the problems, the size of the model can be effectively reduced by quantizing the model from a floating point type to a fixed point type, the model reasoning speed is improved, and the supported embedded equipment types are increased.
Disclosure of Invention
The invention aims to provide a pedestrian detection reasoning method based on embedded equipment, which ensures the precision of a model and improves the reasoning speed of the model by calculating a quantization factor in advance.
In order to achieve the purpose, the invention is realized by the following technical scheme:
1. a pedestrian detection inference method based on embedded equipment is characterized by comprising the following steps:
1) acquiring training data and training a pedestrian detection model;
2) calculating a quantization factor of the model weight, and calculating the quantization factor of the model weight based on the quantization range by calculating the maximum value of the absolute value of the model weight;
3) calculating to obtain an activation value quantization factor of each layer by minimizing the mean square error, calculating the mean square error of quantized output and unquantized output of each layer based on part of test data sets, and obtaining the activation value quantization factor by minimizing the mean square error;
4) quantizing each operator in the model, quantizing the weight of the floating point type model into an int8 data type in an asymmetric quantization mode, and quantizing the activation value into an agent 8 data type;
5) performing model reasoning and inverse quantization, performing model reasoning by using the quantized weight and activation value of the fixed point type, and performing inverse quantization on a reasoning result to obtain an int32 data type;
6) compiling and running the model on the MCU development board.
Preferably, the mean square error formula is as follows:
α＝r/255
in the formula: y is i And
respectively representing unquantized output and quantized output, quantized range r>0, α denotes the quantization factor, clip denotes clipping the activation value to [ -r, r]Ranging, rounding refers to approximating floating point numbers to the nearest integer.
Preferably, the pedestrian detection model adopts a lightweight network MobileNet V1-SSD.
Preferably, the quantization range of the quantization factor is [ -128,127 ].
The invention has the advantages that: the invention obtains the activation value quantization factor of each layer by minimizing the mean square error. The method ensures the precision of the model, improves the reasoning speed of the model by calculating the quantization factor in advance, and can be applied to pedestrian detection reasoning. In addition, the pedestrian detection inference model is operated on the MCU development board, so that the power consumption of the model is reduced.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention.
FIG. 1 is a schematic view of the flow structure of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
A pedestrian detection inference method based on embedded equipment is used for operating a computation-intensive pedestrian detection model on low-power-consumption embedded equipment. The method is characterized in that RISC-V architecture-based edge end deep learning equipment is adopted, an MCU development board of Xilinx is used as a hardware platform, a Wujian100 open source IP is used as an MCU core, an onboard serial port, an HDMI interface and an OV5640 camera are arranged, real-time image data can be captured through the camera, a pedestrian detection inference model is operated on the MCU, and a detection result is output to peripheral equipment through a rear serial port and the HDMI.
1) Training data are obtained, a pedestrian detection model is trained, and a lightweight network MobileNet V1-SSD is adopted in the model.
2) Calculating a quantization factor of the model weight, calculating the quantization factor of the model weight based on a quantization range by calculating the maximum value of the absolute value of the model weight, and quantizing the model weight into int8 type, so that the quantization range is [ -128,127 ];
3) and calculating to obtain an activation value quantization factor of each layer by minimizing the mean square error, calculating the mean square error of quantized output and unquantized output of each layer based on part of the test data set, and obtaining the activation value quantization factor by minimizing the mean square error. As followsIs the mean square error formula, y i And
respectively representing an unquantized output and a quantized output. Quantization Range r (r)>0) The quantization factor α, clip refers to clipping the activation value to [ -r, r]Ranging, rounding refers to approximating floating point numbers to the nearest integer.
α＝r/255
4) And quantizing each operator in the model, quantizing the weight of the floating point type model into an int8 data type in an asymmetric quantization mode, and quantizing the activation value into an agent 8 data type.
5) And performing model reasoning and inverse quantization, performing model reasoning by using the quantized weight and the quantized activation value of the fixed point type, and inversely quantizing a reasoning result into an int32 data type, wherein an activation value quantization factor and an inverse quantization factor participate in calculation in a shifting mode, so that floating point number calculation is avoided.
6) Compiling and operating the model on the MCU development board, preprocessing image data captured by the camera and then transmitting the preprocessed image data to the model, and outputting a pedestrian detection result to peripheral equipment through a rear serial port and an HDMI.
Claims (4)
1. A pedestrian detection inference method based on embedded equipment is characterized by comprising the following steps:
1) acquiring training data and training a pedestrian detection model;
2) calculating a quantization factor of the model weight, and calculating the quantization factor of the model weight based on the quantization range by calculating the maximum value of the absolute value of the model weight;
3) obtaining an activation value quantization factor of each layer by minimizing mean square error calculation, calculating the mean square error of quantized output and unquantized output of each layer based on part of test data sets, and obtaining the activation value quantization factor by minimizing the mean square error;
4) quantizing each operator in the model, quantizing the weight of the floating point type model into an int8 data type in an asymmetric quantization mode, and quantizing the activation value into an agent 8 data type;
5) performing model reasoning and inverse quantization, performing model reasoning by using the quantized weight and activation value of the fixed point type, and performing inverse quantization on a reasoning result to obtain an int32 data type;
6) compiling and running the model on the MCU development board.
2. The pedestrian detection and inference method based on embedded devices of claim 1, wherein the mean square error formula is as follows:
α＝r/255
in the formula: y is i And
respectively representing unquantized output and quantized output, quantized range r>0, alpha denotes the quantization factor, clip denotes the clipping of the activation value to [ -r,r]Ranging, rounding refers to approximating floating point numbers to the nearest integer.
3. The pedestrian detection inference method based on embedded devices according to claim 1, wherein the pedestrian detection model employs a lightweight network MobileNetV 1-SSD.
4. The embedded device-based pedestrian detection inference method of claim 1, wherein a quantization range of the quantization factor is [ -128,127 ].
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210512803.XA CN114821660A (en) | 2022-05-12 | 2022-05-12 | Pedestrian detection inference method based on embedded equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210512803.XA CN114821660A (en) | 2022-05-12 | 2022-05-12 | Pedestrian detection inference method based on embedded equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114821660A true CN114821660A (en) | 2022-07-29 |
Family
ID=82513753
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210512803.XA Withdrawn CN114821660A (en) | 2022-05-12 | 2022-05-12 | Pedestrian detection inference method based on embedded equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114821660A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111950716A (en) * | 2020-08-25 | 2020-11-17 | 云知声智能科技股份有限公司 | Quantification method and system for optimizing int8 |
| CN111950715A (en) * | 2020-08-24 | 2020-11-17 | 云知声智能科技股份有限公司 | 8-bit integer full-quantization inference method and device based on self-adaptive dynamic shift |
| CN112926415A (en) * | 2021-02-05 | 2021-06-08 | 西安电子科技大学 | Pedestrian avoiding system and pedestrian monitoring method |
| CN113947177A (en) * | 2020-07-15 | 2022-01-18 | 安徽寒武纪信息科技有限公司 | Quantization calibration method, calculation device and computer readable storage medium |
| CN114021691A (en) * | 2021-10-13 | 2022-02-08 | 山东浪潮科学研究院有限公司 | Neural network model quantification method, system, device and computer readable medium |
| CN114418062A (en) * | 2021-12-25 | 2022-04-29 | 山东云海国创云计算装备产业创新中心有限公司 | Method, system, device and storage medium for deep convolutional neural network quantization |
-
2022
- 2022-05-12 CN CN202210512803.XA patent/CN114821660A/en not_active Withdrawn
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113947177A (en) * | 2020-07-15 | 2022-01-18 | 安徽寒武纪信息科技有限公司 | Quantization calibration method, calculation device and computer readable storage medium |
| CN111950715A (en) * | 2020-08-24 | 2020-11-17 | 云知声智能科技股份有限公司 | 8-bit integer full-quantization inference method and device based on self-adaptive dynamic shift |
| CN111950716A (en) * | 2020-08-25 | 2020-11-17 | 云知声智能科技股份有限公司 | Quantification method and system for optimizing int8 |
| CN112926415A (en) * | 2021-02-05 | 2021-06-08 | 西安电子科技大学 | Pedestrian avoiding system and pedestrian monitoring method |
| CN114021691A (en) * | 2021-10-13 | 2022-02-08 | 山东浪潮科学研究院有限公司 | Neural network model quantification method, system, device and computer readable medium |
| CN114418062A (en) * | 2021-12-25 | 2022-04-29 | 山东云海国创云计算装备产业创新中心有限公司 | Method, system, device and storage medium for deep convolutional neural network quantization |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109902818B (en) | Distributed acceleration method and system for deep learning training task | |
| WO2019120114A1 (en) | Data fixed point processing method, device, electronic apparatus and computer storage medium | |
| Jiang et al. | Fedmp: Federated learning through adaptive model pruning in heterogeneous edge computing | |
| CN109961147B (en) | Automatic model compression method based on Q-Learning algorithm | |
| CN109663359B (en) | Game intelligent agent training optimization method and device, terminal device and storage medium | |
| CN112435652A (en) | Voice keyword recognition system and method based on graph convolution neural network | |
| CN114021691A (en) | Neural network model quantification method, system, device and computer readable medium | |
| CN108601076A (en) | The multichannel power distribution method of sustainable transmission demand driving in collection of energy wireless network | |
| CN103517079A (en) | Compressed video acquisition and reconstruction system based on data driven subspace set | |
| CN114821660A (en) | Pedestrian detection inference method based on embedded equipment | |
| Gong et al. | A Scalable Multi-Device Semantic Communication System for Multi-Task Execution | |
| CN114998661B (en) | Target detection method based on fixed point quantitative determination | |
| CN116301914A (en) | Convolutional neural network deployment method based on GAP8 microprocessor | |
| CN114821368A (en) | Power defect detection method based on reinforcement learning and Transformer | |
| CN113989102A (en) | Rapid style migration method with high shape-preserving property | |
| CN113079391A (en) | Portrait image mixing processing method, equipment and computer readable storage medium | |
| CN110378958B (en) | Sea surface generation method based on FFT data pre-baking technology | |
| CN117097797B (en) | Cloud edge end cooperation method, device and system, electronic equipment and readable storage medium | |
| CN114926394B (en) | Colorectal cancer pathological image segmentation method based on pixel contrast learning | |
| CN113095472B (en) | Method for reducing precision loss by forward reasoning of convolutional neural network in quantization process | |
| CN112288081A (en) | Convolutional neural network operation system | |
| CN117993459A (en) | Training method and system for noise reduction network model | |
| Sun et al. | Image Coding and Compression Method Based on Semantics of Pixels for Semantic Communication | |
| CN117978247A (en) | Data compression method and device based on high-flux satellite Internet of things | |
| Yu et al. | Design of Programmable Parallel Vision Processor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20220729 |