CN111178322A - Identification acceleration system and method for edge image - Google Patents
Identification acceleration system and method for edge image Download PDFInfo
- Publication number
- CN111178322A CN111178322A CN202010025490.6A CN202010025490A CN111178322A CN 111178322 A CN111178322 A CN 111178322A CN 202010025490 A CN202010025490 A CN 202010025490A CN 111178322 A CN111178322 A CN 111178322A
- Authority
- CN
- China
- Prior art keywords
- module
- image
- mpsoc
- identification
- interface
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Computational Linguistics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses an identification acceleration system and method for an edge image, the system takes FPGA and MPSOC as core processors, wherein in order to meet the real-time requirement of image identification, the FPGA is taken as the core to realize an image identification algorithm, an MPSOC chip comprises a multi-core Processor (PS), a GPU and PL logic, and the MPSOC realizes the functions of monitoring management, video preprocessing, image detection processing selection, image detection algorithm, image matching, image identification preprocessing and the like. The system has strong neural network computing power, does not need to transmit large-scale image information to a cloud for identification, and has the advantages of high real-time performance, strong expandability and low power consumption.
Description
Technical Field
The invention relates to the field of video processing, in particular to a system and a method for identifying and accelerating an edge image.
Background
Most of large-scale image recognition systems adopt a cloud end to process large-scale image recognition algorithms, the system brings larger delay and higher cost, and some edge image recognition systems can only process small-scale image recognition algorithms in real time, so that the energy consumption is high and the real-time performance is poor.
Disclosure of Invention
The invention aims to solve the technical problem of providing a system and a method for identifying and accelerating an edge image, which can realize a large-scale image identification algorithm at an edge.
In order to solve the technical problem, the technical scheme adopted by the invention is as follows: the utility model provides a discernment accelerating system for marginal end image, including MPSOC and FPGA, MPSOC includes the PS end, PL end and high-speed interface, the PS end includes video preprocessing module, the image detection module, face identification preprocessing module, the MTCNN module, face matching module and control management module, the PL end includes face identification preprocessing module and MTCNN module, FPGA includes the control interface, data analysis module and mobilefacenet module, video preprocessing module links to each other with the image detection module, the image detection module links to each other with the face identification preprocessing module of PS end and PL end respectively, face identification preprocessing module links to each other with the MTCNN module, the MTCNN module passes through the data analysis module of high-speed interface with FPGA, data analysis module and mobilefacenet module, mobilefacenet module passes through high-speed interface and face matching module, face matching module links to each other with the DDR that the storage has the image characteristic to link to each other simultaneously.
Furthermore, the number of the FPGAs connected with the MPSOC is more than two.
Further, the high-speed interface is an srio interface or an Aurora interface.
The invention also discloses an identification acceleration method for the edge images, which comprises the following steps: s01), the video preprocessing module preprocesses the video image entering the MPSOC; s02), the image detection module marks the preprocessed video images, evaluates whether the PS terminal processor can meet the required real-time performance according to the number of the images, and inputs the images preprocessed by the video to the face recognition preprocessing module and the MTCNN module of the PL terminal for processing when the next frame of video image is processed if the images can not meet the required real-time performance; s03), dividing the numbered N images into M groups by a face recognition preprocessing module, wherein each group comprises N/M images, M, N is a positive integer, M is more than or equal to 2, and N is more than M, then forming M groups of data frames by the M groups of images according to labels, and respectively sending the M groups of data frames to M FPGAs through high-speed interfaces; s04), the data analysis module in the FPGA strips the image data from the received data frame, and then transmits the image data to the mobility network module; s05) the mobilefacenet module carries out image recognition on the received image data, and the generated image feature vectors are formed into frames with a certain format and sent back to the MPSOC through the high-speed interface; s06), the MPSOC receives the image feature vector frame returned by the mobile facenet module, and the image feature vector frame is unframed and then sent to the face matching module at the PS end; s07), the face matching module compares the image feature vector identified by the mobility facenet module with the image feature vector in the DDR storage database to obtain an image identification result, and the result is displayed through a video.
Furthermore, the number of the FPGAs connected with the MPSOC is more than two.
Further, the high-speed interface is an srio interface or an Aurora interface.
Further, frames sent to the FPGA by the MPSOC and returned to the MPSOC by the FPGA are image data plus a header flag and a crc check bit, and the data parsing module strips the image data by removing the added header flag and crc check bit, so that only the image data is left.
Furthermore, the video preprocessing module cuts and shapes the image.
The invention has the beneficial effects that: the invention takes FPGA and MPSOC as core processors, wherein in order to reach the real-time requirement of image recognition, the FPGA is taken as the core to realize the image recognition algorithm, the MPSOC chip comprises a multi-core Processor (PS), a GPU and PL logic, and the MPSOC realizes the functions of monitoring management, video preprocessing, image detection processing selection, image detection algorithm, image matching, image recognition preprocessing and the like. The system has strong neural network computing power, does not need to transmit large-scale image information to a cloud for identification, and has the advantages of high real-time performance, strong expandability and low power consumption.
Drawings
Fig. 1 is a schematic block diagram of the system described in embodiment 1.
Detailed Description
The invention is further described with reference to the following figures and specific embodiments.
Example 1
The embodiment discloses an identification acceleration system for edge images, which comprises an MPSOC and an FPGA, wherein the MPSOC comprises a PS terminal, a PL terminal and a high-speed interface, the PS terminal comprises a video preprocessing module, an image detection module, a face identification preprocessing module, an MTCNN module, a face matching module and a monitoring management module, the PL terminal comprises a face identification preprocessing module and an MTCNN module, the FPGA comprises a control interface, a data analysis module and a mobilefacenet module, the video preprocessing module is connected with the image detection module, the image detection module is respectively connected with the face recognition preprocessing modules of the PS end and the PL end, the face recognition preprocessing module is connected with the MTCNN module, the MTCNN module is connected with the data analysis module of the FPGA through a high-speed interface, the data analysis module is connected with the mobility module, the mobility module is connected with the face matching module through a high-speed interface, and the face matching module is simultaneously connected with the DDR (double data rate) which stores image features and is connected with the DDR.
In this embodiment, the number of FPGAs connected to the MPSOC is two or more, and the number of connected FPGAs is determined according to the number of images to be processed. The system can be expanded according to the number of the predicted images, more FPGAs are added to accelerate the larger-scale image recognition algorithm, and the real-time performance of image recognition is guaranteed.
In this embodiment, the high-speed interface is an srio interface or an Aurora interface.
Example 2
The embodiment discloses an identification acceleration method for an edge image, which comprises the following steps: s01), the video preprocessing module preprocesses the video image entering the MPSOC; s02), the image detection module marks the preprocessed video images, evaluates whether the PS terminal processor can meet the required real-time performance according to the number of the images, and inputs the images preprocessed by the video to the face recognition preprocessing module and the MTCNN module of the PL terminal for processing when the next frame of video image is processed if the images can not meet the required real-time performance; s03), the face recognition module divides the numbered N images into M groups, each group comprises N/M images, M, N is a positive integer, M is more than or equal to 2, N is more than M, then the M groups of images form M groups of data frames according to the labels, and the M groups of data frames are respectively sent to M FPGAs through high-speed interfaces; s04), the data analysis module in the FPGA strips the image data from the received data frame, and then transmits the image data to the mobility network module; s05) the mobilefacenet module carries out image recognition on the received image data, and the generated image feature vectors are formed into frames with a certain format and sent back to the MPSOC through the high-speed interface; s06), the MPSOC receives the image feature vector frame returned by the mobile facenet module, and the image feature vector frame is unframed and then sent to the face matching module at the PS end; s07), the face matching module compares the image feature vector identified by the mobility facenet module with the image feature vector in the DDR storage database to obtain an image identification result, and the result is displayed through a video.
In this embodiment, the number of FPGAs connected to the MPSOC is two or more.
In this embodiment, the high-speed interface is an srio interface or an Aurora interface.
In this embodiment, frames sent by the MPSOC to the FPGA and returned by the FPGA to the MPSOC are image data plus a header flag and a crc check bit, and the data parsing module strips the image data is to remove the added header flag and crc check bit, so that only the image data is left.
In this embodiment, the video preprocessing module performs clipping and shaping on the image.
The invention takes FPGA and MPSOC as core processors, wherein in order to reach the real-time requirement of image recognition, the FPGA is taken as the core to realize the image recognition algorithm, the MPSOC chip comprises a multi-core Processor (PS), a GPU and PL logic, and the MPSOC realizes the functions of monitoring management, video preprocessing, image detection processing selection, image detection algorithm, image matching, image recognition preprocessing and the like. The system has strong neural network computing power, does not need to transmit large-scale image information to a cloud for identification, and has the advantages of high real-time performance, strong expandability and low power consumption.
The foregoing description is only for the basic principle and the preferred embodiments of the present invention, and modifications and substitutions by those skilled in the art are included in the scope of the present invention.
Claims (8)
1. An identification acceleration system for an edge-end image, characterized by: the system comprises an MPSOC and an FPGA, the MPSOC comprises a PS end, a PL end and a high-speed interface, the PS end comprises a video preprocessing module, an image detection module, a face recognition preprocessing module, an MTCNN module, a face matching module and a monitoring management module, the PL end comprises the face recognition preprocessing module and the MTCNN module, the FPGA comprises a control interface, a data analysis module and a mobility interface module, the video preprocessing module is connected with the image detection module, the image detection module is respectively connected with the face recognition preprocessing module of the PS end and the PL end, the face recognition preprocessing module is connected with the MTCNN module, the MTCNN module is connected with the data analysis module of the FPGA through the high-speed interface, the data analysis module is connected with the mobility interface module, the mobility interface module is connected with the face matching module through the high-speed interface, and the face matching module is simultaneously connected with a DDR stored with image features.
2. An identification acceleration system for an edge-end image according to claim 1, characterized in that: the number of the FPGAs connected with the MPSOC is more than two.
3. An identification acceleration system for an edge-end image according to claim 1, characterized in that: the high-speed interface is an srio interface or an Aurora interface.
4. A method for accelerating the identification of an edge image is characterized in that: the method comprises the following steps: s01), the video preprocessing module preprocesses the video image entering the MPSOC; s02), the image detection module marks the preprocessed video images, evaluates whether the PS terminal processor can meet the required real-time performance according to the number of the images, and inputs the images preprocessed by the video to the face recognition preprocessing module and the MTCNN module of the PL terminal for processing when the next frame of video image is processed if the images can not meet the required real-time performance; s03), dividing the numbered N images into M groups by a face recognition preprocessing module, wherein each group comprises N/M images, M, N is a positive integer, M is more than or equal to 2, and N is more than M, then forming M groups of data frames by the M groups of images according to labels, and respectively sending the M groups of data frames to M FPGAs through high-speed interfaces; s04), the data analysis module in the FPGA strips the image data from the received data frame, and then transmits the image data to the mobility network module; s05) the mobilefacenet module carries out image recognition on the received image data, and the generated image feature vectors are formed into frames with a certain format and sent back to the MPSOC through the high-speed interface; s06), the MPSOC receives the image feature vector frame returned by the mobile facenet module, and the image feature vector frame is unframed and then sent to the face matching module at the PS end; s07), the face matching module compares the image feature vector identified by the mobility facenet module with the image feature vector in the DDR storage database to obtain an image identification result, and the result is displayed through a video.
5. The identification acceleration method for the edge end image according to claim 4, characterized in that: the number of the FPGAs connected with the MPSOC is more than two.
6. The identification acceleration method for the edge end image according to claim 4, characterized in that: the high-speed interface is an srio interface or an Aurora interface.
7. The identification acceleration method for the edge end image according to claim 4, characterized in that: frames sent to the FPGA by the MPSOC and returned to the MPSOC by the FPGA are image data, a message header mark and a crc check bit are added to the image data, and the data analysis module strips the image data, namely the added message header mark and the crc check bit are removed, so that only the image data are left.
8. The identification acceleration method for the edge end image according to claim 4, characterized in that: the video preprocessing module is used for cutting and shaping images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010025490.6A CN111178322B (en) | 2020-01-10 | 2020-01-10 | Identification acceleration system and method for edge image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010025490.6A CN111178322B (en) | 2020-01-10 | 2020-01-10 | Identification acceleration system and method for edge image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111178322A true CN111178322A (en) | 2020-05-19 |
| CN111178322B CN111178322B (en) | 2022-09-30 |
Family
ID=70652659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010025490.6A Active CN111178322B (en) | 2020-01-10 | 2020-01-10 | Identification acceleration system and method for edge image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111178322B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116559789A (en) * | 2023-07-07 | 2023-08-08 | 成都泰格微电子研究所有限责任公司 | Signal processing method of radar control system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105631798A (en) * | 2016-03-04 | 2016-06-01 | 北京理工大学 | Low-power consumption portable real-time image target detecting and tracking system and method thereof |
| CN205486304U (en) * | 2016-03-04 | 2016-08-17 | 北京理工大学 | Portable realtime graphic object detection of low -power consumption and tracking means |
| CN206312166U (en) * | 2016-09-18 | 2017-07-07 | 常州机电职业技术学院 | Robot vision image preprocessing system and robot |
| CN106937052A (en) * | 2017-03-29 | 2017-07-07 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN107766812A (en) * | 2017-10-12 | 2018-03-06 | 东南大学—无锡集成电路技术研究所 | A kind of real-time face detection identifying system based on MiZ702N |
| CN109389120A (en) * | 2018-10-29 | 2019-02-26 | 济南浪潮高新科技投资发展有限公司 | A kind of object detecting device based on zynqMP |
| CN110399824A (en) * | 2019-07-19 | 2019-11-01 | 苏州浪潮智能科技有限公司 | Zynq system and images steganalysis method based on zynq system |
-
2020
- 2020-01-10 CN CN202010025490.6A patent/CN111178322B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105631798A (en) * | 2016-03-04 | 2016-06-01 | 北京理工大学 | Low-power consumption portable real-time image target detecting and tracking system and method thereof |
| CN205486304U (en) * | 2016-03-04 | 2016-08-17 | 北京理工大学 | Portable realtime graphic object detection of low -power consumption and tracking means |
| CN206312166U (en) * | 2016-09-18 | 2017-07-07 | 常州机电职业技术学院 | Robot vision image preprocessing system and robot |
| CN106937052A (en) * | 2017-03-29 | 2017-07-07 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN107766812A (en) * | 2017-10-12 | 2018-03-06 | 东南大学—无锡集成电路技术研究所 | A kind of real-time face detection identifying system based on MiZ702N |
| CN109389120A (en) * | 2018-10-29 | 2019-02-26 | 济南浪潮高新科技投资发展有限公司 | A kind of object detecting device based on zynqMP |
| CN110399824A (en) * | 2019-07-19 | 2019-11-01 | 苏州浪潮智能科技有限公司 | Zynq system and images steganalysis method based on zynq system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116559789A (en) * | 2023-07-07 | 2023-08-08 | 成都泰格微电子研究所有限责任公司 | Signal processing method of radar control system |
| CN116559789B (en) * | 2023-07-07 | 2023-09-19 | 成都泰格微电子研究所有限责任公司 | Signal processing method of radar control system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111178322B (en) | 2022-09-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103177262B (en) | Based on FPGA framework and its implementation of the pedestrian detecting system of HOG and SVM | |
| CN104182748B (en) | One kind is based on the matched Chinese-character stroke extraction method of fractionation | |
| CN111178322B (en) | Identification acceleration system and method for edge image | |
| CN111614769A (en) | Intelligent behavior analysis engine system of deep learning technology and control method | |
| CN112163045A (en) | Block chain-based edge data secure storage system | |
| CN113691305B (en) | Cloud-edge-end-based picture transmission method for Beidou third-number short message channel | |
| CN104519059B (en) | The Real-time Alarm processing system and method for a kind of vehicle | |
| CN106341660A (en) | Image processing method and apparatus thereof, and unmanned aerial vehicle image processing system | |
| CN112019500A (en) | Encrypted traffic identification method based on deep learning and electronic device | |
| CN111310488B (en) | Self-adaptive M-system tree RFID anti-collision method | |
| CN117593623A (en) | Lightweight vehicle detection method based on improved YOLOv8n model | |
| CN112561902A (en) | Chip inverse reduction method and system based on deep learning | |
| US11954955B2 (en) | Method and system for collecting and monitoring vehicle status information | |
| CN102340372A (en) | Method and device for increasing bit throughput at transmitting end of LTE (Long Term Evolution) base station | |
| CN115830533A (en) | Helmet wearing detection method based on K-means clustering improved YOLOv5 algorithm | |
| Xie et al. | Mask Wearing Detection Based on YOLOv5 Target Detection Algorithm under COVID-19'' | |
| CN114283442A (en) | Intelligent identification method and device for secondary wiring diagram and storage medium | |
| CN116546439A (en) | Method and system for quickly waking up electronic shelf label and sending group message | |
| CN106230989A (en) | Location data syn-chronization real-time continuous produced is to the method and system of server | |
| CN112738225A (en) | Edge calculation method based on artificial intelligence | |
| CN116108492B (en) | Laterally expandable data leakage prevention system | |
| CN112910688B (en) | OCSVM model-based communication behavior abnormal parallel detection method and system under HJ212 protocol | |
| CN114979017B (en) | Deep learning protocol identification method and system based on original flow of industrial control system | |
| CN114978977A (en) | Edge computing gateway with multi-data fusion processing | |
| CN111401317B (en) | Video classification method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220825 Address after: 250000 building S02, No. 1036, Gaoxin Inspur Road, Jinan, Shandong Applicant after: Shandong Inspur Scientific Research Institute Co.,Ltd. Address before: 250104 1st floor, R & D building, No. 2877, Suncun Town, Licheng District, Jinan City, Shandong Province Applicant before: JINAN INSPUR HIGH-TECH TECHNOLOGY DEVELOPMENT Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A recognition acceleration system and method for edge images Effective date of registration: 20230613 Granted publication date: 20220930 Pledgee: Qilu Bank Co.,Ltd. Jinan Science and Technology Innovation Financial Center Branch Pledgor: Shandong Inspur Scientific Research Institute Co.,Ltd. Registration number: Y2023980043550 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |