CN111178322B - Identification acceleration system and method for edge image - Google Patents
Identification acceleration system and method for edge image Download PDFInfo
- Publication number
- CN111178322B CN111178322B CN202010025490.6A CN202010025490A CN111178322B CN 111178322 B CN111178322 B CN 111178322B CN 202010025490 A CN202010025490 A CN 202010025490A CN 111178322 B CN111178322 B CN 111178322B
- Authority
- CN
- China
- Prior art keywords
- image
- module
- mpsoc
- images
- identification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
Abstract
The invention discloses an identification acceleration system and method for an edge image, the system takes FPGA and MPSOC as core processors, wherein in order to meet the real-time requirement of image identification, the FPGA is taken as the core to realize an image identification algorithm, an MPSOC chip comprises a multi-core Processor (PS), a GPU and PL logic, and the MPSOC realizes the functions of monitoring management, video preprocessing, image detection processing selection, image detection algorithm, image matching, image identification preprocessing and the like. The system has strong neural network computing power, does not need to transmit large-scale image information to a cloud for identification, and has the advantages of high real-time performance, strong expandability and low power consumption.
Description
Technical Field
The invention relates to the field of video processing, in particular to a system and a method for identifying and accelerating an edge image.
Background
Most of large-scale image recognition systems adopt a cloud end to process large-scale image recognition algorithms, the system brings larger delay and higher cost, and some edge image recognition systems can only process small-scale image recognition algorithms in real time, so that the energy consumption is high and the real-time performance is poor.
Disclosure of Invention
The invention aims to solve the technical problem of providing a system and a method for identifying and accelerating an edge image, which can realize a large-scale image identification algorithm at an edge.
In order to solve the technical problem, the technical scheme adopted by the invention is as follows: the utility model provides a discernment accelerating system for marginal end image, including MPSOC and FPGA, MPSOC includes the PS end, PL end and high-speed interface, the PS end includes video preprocessing module, the image detection module, face identification preprocessing module, the MTCNN module, face matching module and control management module, the PL end includes face identification preprocessing module and MTCNN module, FPGA includes the control interface, data analysis module and mobilefacenet module, video preprocessing module links to each other with the image detection module, the image detection module links to each other with the face identification preprocessing module of PS end and PL end respectively, face identification preprocessing module links to each other with the MTCNN module, the MTCNN module passes through the data analysis module of high-speed interface with FPGA, data analysis module and mobilefacenet module, mobilefacenet module passes through high-speed interface and face matching module, face matching module links to each other with the DDR that the storage has the image characteristic to link to each other simultaneously.
Furthermore, the number of the FPGAs connected with the MPSOC is more than two.
Further, the high-speed interface is an srio interface or an Aurora interface.
The invention also discloses an identification acceleration method for the edge images, which comprises the following steps: s01), the video preprocessing module preprocesses the video image entering the MPSOC; s02), the image detection module marks the preprocessed video images, evaluates whether the PS terminal processor can meet the required real-time performance according to the number of the images, and inputs the images preprocessed by the video to the face recognition preprocessing module and the MTCNN module of the PL terminal for processing when the next frame of video image is processed if the images can not meet the required real-time performance; s03), dividing the numbered N images into M groups by a face recognition preprocessing module, wherein each group comprises N/M images, M, N is a positive integer, M is more than or equal to 2, and N is more than M, then forming M groups of data frames by the M groups of images according to labels, and respectively sending the M groups of data frames to M FPGAs through high-speed interfaces; s04), the data analysis module in the FPGA strips the image data from the received data frame, and then transmits the image data to the mobility network module; s05) the mobilefacenet module carries out image recognition on the received image data, and the generated image feature vectors are formed into frames with a certain format and sent back to the MPSOC through the high-speed interface; s06), the MPSOC receives the image feature vector frame returned by the mobility network module, and the image feature vector frame is sent to a face matching module at the PS end after being unframed; s07), the face matching module compares the image feature vector identified by the mobility facenet module with the image feature vector in the DDR storage database to obtain an image identification result, and the result is displayed through a video.
Furthermore, the number of the FPGAs connected with the MPSOC is more than two.
Further, the high-speed interface is an srio interface or an Aurora interface.
Further, frames sent to the FPGA by the MPSOC and returned to the MPSOC by the FPGA are image data plus a header flag and a crc check bit, and the data parsing module strips the image data by removing the added header flag and crc check bit, so that only the image data is left.
Furthermore, the video preprocessing module cuts and shapes the image.
The invention has the beneficial effects that: the invention takes FPGA and MPSOC as core processors, wherein in order to reach the real-time requirement of image recognition, the FPGA is taken as the core to realize the image recognition algorithm, the MPSOC chip comprises a multi-core Processor (PS), a GPU and PL logic, and the MPSOC realizes the functions of monitoring management, video preprocessing, image detection processing selection, image detection algorithm, image matching, image recognition preprocessing and the like. The system has strong neural network computing power, does not need to transmit large-scale image information to a cloud for identification, and has the advantages of high real-time performance, strong expandability and low power consumption.
Drawings
Fig. 1 is a schematic block diagram of the system described in embodiment 1.
Detailed Description
The invention is further described with reference to the following figures and specific examples.
Example 1
The embodiment discloses an identification acceleration system for edge images, which comprises an MPSOC and an FPGA, wherein the MPSOC comprises a PS terminal, a PL terminal and a high-speed interface, the PS terminal comprises a video preprocessing module, an image detection module, a face identification preprocessing module, an MTCNN module, a face matching module and a monitoring management module, the PL terminal comprises a face identification preprocessing module and an MTCNN module, the FPGA comprises a control interface, a data analysis module and a mobilefacenet module, the video preprocessing module is connected with the image detection module, the image detection module is respectively connected with the face recognition preprocessing modules of the PS end and the PL end, the face recognition preprocessing module is connected with the MTCNN module, the MTCNN module is connected with the data analysis module of the FPGA through a high-speed interface, the data analysis module is connected with the mobility module, the mobility module is connected with the face matching module through a high-speed interface, and the face matching module is simultaneously connected with the DDR (double data rate) which stores image features and is connected with the DDR.
In this embodiment, the number of FPGAs connected to the MPSOC is two or more, and the number of connected FPGAs is determined according to the number of images to be processed. The system can be expanded according to the number of the predicted images, more FPGAs are added to accelerate the larger-scale image recognition algorithm, and the real-time performance of image recognition is guaranteed.
In this embodiment, the high-speed interface is an srio interface or an Aurora interface.
Example 2
The embodiment discloses an identification acceleration method for an edge image, which comprises the following steps: s01), the video preprocessing module preprocesses the video image entering the MPSOC; s02), the image detection module marks the preprocessed video images, evaluates whether the PS terminal processor can meet the required real-time performance according to the number of the images, and inputs the images preprocessed by the video to the face recognition preprocessing module and the MTCNN module of the PL terminal for processing when the next frame of video image is processed if the images can not meet the required real-time performance; s03), the face recognition module divides the numbered N images into M groups, each group comprises N/M images, M, N is a positive integer, M is more than or equal to 2, N is more than M, then the M groups of images form M groups of data frames according to the labels, and the M groups of data frames are respectively sent to M FPGAs through high-speed interfaces; s04), the data analysis module in the FPGA strips the image data from the received data frame, and then transmits the image data to the mobility module; s05), the mobilefacetet module carries out image recognition to the received image data, and forms the generated image characteristic vector into a frame with a certain format and sends the frame back to the MPSOC through the high-speed interface; s06), the MPSOC receives the image feature vector frame returned by the mobility network module, and the image feature vector frame is sent to a face matching module at the PS end after being unframed; s07), the face matching module compares the image characteristic vector identified by the mobility network module with the image characteristic vector in the database stored in the DDR to obtain the image identification result, and the result is displayed through video.
In this embodiment, the number of FPGAs connected to the MPSOC is two or more.
In this embodiment, the high-speed interface is an srio interface or an Aurora interface.
In this embodiment, frames sent by the MPSOC to the FPGA and returned by the FPGA to the MPSOC are both image data plus a header flag of the message and a crc check bit, and the data parsing module strips the image data by removing the header flag of the message and the crc check bit that are added, so as to leave only the image data.
In this embodiment, the video preprocessing module performs clipping and shaping on the image.
The invention takes FPGA and MPSOC as core processors, wherein in order to achieve the real-time requirement of image recognition, the FPGA is taken as the core to realize the image recognition algorithm, the MPSOC chip comprises a multi-core Processor (PS), a GPU and PL logic, and the MPSOC realizes the functions of monitoring management, video preprocessing, image detection processing selection, image detection algorithm, image matching, image recognition preprocessing and the like. The system has strong neural network computing power, does not need to transmit large-scale image information to a cloud for identification, and has the advantages of high real-time performance, strong expandability and low power consumption.
The foregoing description is only for the purpose of illustrating the general principles and preferred embodiments of the present invention, and it is intended that modifications and substitutions be made by those skilled in the art in light of the present invention and that they fall within the scope of the present invention.
Claims (5)
1. A method for accelerating the identification of an edge image is characterized in that: the method comprises the following steps: s01), the video preprocessing module preprocesses the video image entering the MPSOC; s02), the image detection module marks the preprocessed video images, evaluates whether the PS terminal processor can meet the real-time performance of the requirement according to the number of the images, and inputs the images preprocessed by the video into the face recognition preprocessing module and the MTCNN module of the PL terminal for processing when the next frame of video image is processed if the images can not meet the requirement; s03), dividing the numbered N images into M groups by a face recognition preprocessing module, wherein each group comprises N/M images, M, N is a positive integer, M is more than or equal to 2, and N is more than M, then forming M groups of data frames by the M groups of images according to the labels, and respectively sending the M groups of data frames to M FPGAs through high-speed interfaces; s04), the data analysis module in the FPGA strips the image data from the received data frame, and then transmits the image data to the mobility network module; s05), the mobilefacetet module carries out image recognition to the received image data, and forms the generated image characteristic vector into a frame with a certain format and sends the frame back to the MPSOC through the high-speed interface; s06), the MPSOC receives the image feature vector frame returned by the mobile facenet module, and the image feature vector frame is unframed and then sent to the face matching module at the PS end; s07), the face matching module compares the image feature vector identified by the mobility facenet module with the image feature vector in the DDR storage database to obtain an image identification result, and the result is displayed through a video.
2. The identification acceleration method for the edge-end image according to claim 1, characterized in that: the number of the FPGAs connected with the MPSOC is more than two.
3. The identification acceleration method for the edge end image according to claim 1, characterized in that: the high-speed interface is an srio interface or an Aurora interface.
4. The identification acceleration method for the edge end image according to claim 1, characterized in that: frames sent by the MPSOC to the FPGA and returned to the MPSOC by the FPGA are image data plus a message header mark and a crc check bit, and the data analysis module strips the image data and removes the added message header mark and the crc check bit to only leave the image data.
5. The identification acceleration method for the edge end image according to claim 1, characterized in that: the video preprocessing module is used for cutting and shaping images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010025490.6A CN111178322B (en) | 2020-01-10 | 2020-01-10 | Identification acceleration system and method for edge image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010025490.6A CN111178322B (en) | 2020-01-10 | 2020-01-10 | Identification acceleration system and method for edge image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111178322A CN111178322A (en) | 2020-05-19 |
| CN111178322B true CN111178322B (en) | 2022-09-30 |
Family
ID=70652659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010025490.6A Active CN111178322B (en) | 2020-01-10 | 2020-01-10 | Identification acceleration system and method for edge image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111178322B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116559789B (en) * | 2023-07-07 | 2023-09-19 | 成都泰格微电子研究所有限责任公司 | Signal processing method of radar control system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105631798A (en) * | 2016-03-04 | 2016-06-01 | 北京理工大学 | Low-power consumption portable real-time image target detecting and tracking system and method thereof |
| CN205486304U (en) * | 2016-03-04 | 2016-08-17 | 北京理工大学 | Portable realtime graphic object detection of low -power consumption and tracking means |
| CN206312166U (en) * | 2016-09-18 | 2017-07-07 | 常州机电职业技术学院 | Robot vision image preprocessing system and robot |
| CN106937052A (en) * | 2017-03-29 | 2017-07-07 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN107766812A (en) * | 2017-10-12 | 2018-03-06 | 东南大学—无锡集成电路技术研究所 | A kind of real-time face detection identifying system based on MiZ702N |
| CN109389120A (en) * | 2018-10-29 | 2019-02-26 | 济南浪潮高新科技投资发展有限公司 | A kind of object detecting device based on zynqMP |
| CN110399824A (en) * | 2019-07-19 | 2019-11-01 | 苏州浪潮智能科技有限公司 | Zynq system and images steganalysis method based on zynq system |
-
2020
- 2020-01-10 CN CN202010025490.6A patent/CN111178322B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105631798A (en) * | 2016-03-04 | 2016-06-01 | 北京理工大学 | Low-power consumption portable real-time image target detecting and tracking system and method thereof |
| CN205486304U (en) * | 2016-03-04 | 2016-08-17 | 北京理工大学 | Portable realtime graphic object detection of low -power consumption and tracking means |
| CN206312166U (en) * | 2016-09-18 | 2017-07-07 | 常州机电职业技术学院 | Robot vision image preprocessing system and robot |
| CN106937052A (en) * | 2017-03-29 | 2017-07-07 | 维沃移动通信有限公司 | The processing method and mobile terminal of a kind of view data |
| CN107766812A (en) * | 2017-10-12 | 2018-03-06 | 东南大学—无锡集成电路技术研究所 | A kind of real-time face detection identifying system based on MiZ702N |
| CN109389120A (en) * | 2018-10-29 | 2019-02-26 | 济南浪潮高新科技投资发展有限公司 | A kind of object detecting device based on zynqMP |
| CN110399824A (en) * | 2019-07-19 | 2019-11-01 | 苏州浪潮智能科技有限公司 | Zynq system and images steganalysis method based on zynq system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111178322A (en) | 2020-05-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102194012B (en) | Microblog topic detecting method and system | |
| CN111178322B (en) | Identification acceleration system and method for edge image | |
| CN103177262A (en) | FPGA (field programmable gate array) architecture of HOG (histogram of oriented gradient) and SVM (support vector machine) based pedestrian detection system and implementing method of FPGA architecture | |
| CN111614769A (en) | Intelligent behavior analysis engine system of deep learning technology and control method | |
| CN113691305B (en) | Cloud-edge-end-based picture transmission method for Beidou third-number short message channel | |
| CN112163045A (en) | Block chain-based edge data secure storage system | |
| CN114092742A (en) | Small sample image classification device and method based on multiple angles | |
| CN110738148A (en) | Cloud target detection algorithm based on heterogeneous platform | |
| CN110414431B (en) | Face recognition method and system based on elastic context relation loss function | |
| CN104519059B (en) | The Real-time Alarm processing system and method for a kind of vehicle | |
| CN111310488B (en) | Self-adaptive M-system tree RFID anti-collision method | |
| CN117593623A (en) | Lightweight vehicle detection method based on improved YOLOv8n model | |
| CN112561902A (en) | Chip inverse reduction method and system based on deep learning | |
| CN111857734A (en) | Deployment and use method of distributed deep learning model platform | |
| US11954955B2 (en) | Method and system for collecting and monitoring vehicle status information | |
| Shakiba et al. | Fitted dynamic framed slotted ALOHA anti-collision algorithm in RFID systems | |
| CN116546439B (en) | Method and system for quickly waking up electronic shelf label and sending group message | |
| CN115830533A (en) | Helmet wearing detection method based on K-means clustering improved YOLOv5 algorithm | |
| CN114283442A (en) | Intelligent identification method and device for secondary wiring diagram and storage medium | |
| CN106230989A (en) | Location data syn-chronization real-time continuous produced is to the method and system of server | |
| CN112738225A (en) | Edge calculation method based on artificial intelligence | |
| CN113794601B (en) | Network traffic processing method, device and computer readable storage medium | |
| CN112910688B (en) | OCSVM model-based communication behavior abnormal parallel detection method and system under HJ212 protocol | |
| Yue et al. | Pedestrian action recognition in infrared image based on improved GoogLeNet | |
| CN114978977A (en) | Edge computing gateway with multi-data fusion processing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220825 Address after: 250000 building S02, No. 1036, Gaoxin Inspur Road, Jinan, Shandong Applicant after: Shandong Inspur Scientific Research Institute Co.,Ltd. Address before: 250104 1st floor, R & D building, No. 2877, Suncun Town, Licheng District, Jinan City, Shandong Province Applicant before: JINAN INSPUR HIGH-TECH TECHNOLOGY DEVELOPMENT Co.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A recognition acceleration system and method for edge images Effective date of registration: 20230613 Granted publication date: 20220930 Pledgee: Qilu Bank Co.,Ltd. Jinan Science and Technology Innovation Financial Center Branch Pledgor: Shandong Inspur Scientific Research Institute Co.,Ltd. Registration number: Y2023980043550 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |