TWI723383B - Image recognition device and method - Google Patents

Abstract

An image recognition method includes: obtaining a video stream by a first processing module; pre-processing the video stream to obtain an image queue arranged in a frame play order by the first processing module; storing the image queue into a storage module; reading a image frame of the image queue from the storage module and identifying the image frame to detect at least one target in the image frame by the second processing module; wherein the second processing module sequentially reads the image frame one by one according to a frame order of the image queue. An image recognition device is also provided.

Description

Image recognition device and method

The present invention relates to the field of image processing technology, in particular to an image recognition device and method.

With the development of high-definition display technology, 4K (picture element resolution 4096*2160) video playback equipment appeared on the market, and 8K (picture element resolution 7680*4320) video playback equipment may even be marketed in the future. The frame rate of these video playback devices is generally 30FPS or higher. When it is necessary to detect one or more target objects in each frame of image, due to the higher image elements of the video frame and the larger frame rate, the image recognition workload is larger and the recognition speed is required to be fast. The conventional image The processing speed of the processor of the recognition module is limited, and it is unable to perform real-time target detection for each UHD frame image played.

In view of this, it is necessary to provide an image recognition device and method that can improve the efficiency of image recognition and realize real-time target detection for each ultra-high-definition frame image.

An embodiment of the present invention provides an image recognition method applied to an image recognition device. The image recognition device includes a first processing module, a storage module, and a second processing module. The method includes: using the The first processing module obtains the video stream to be processed, and preprocesses the video stream to obtain an image queue arranged in the order of frame playback; stores the image queue in the storage module; and uses The second processing module reads the image frame to be recognized from the storage module, and compares the image The frame is recognized to detect at least one target in the image frame; wherein, the second processing module reads sequentially according to the sequence of the image frames in the image queue.

An embodiment of the present invention also provides an image recognition device, including a first processing module, a storage module, a second processing module, and a communication bus. The first processing module, the storage module, and the The second processing module communicates with each other through the communication bus; the first processing module is used to obtain the to-be-processed video stream, and preprocess the video stream to obtain the order of frame playback Image queue; the first processing module is also used to store the image queue to the storage module; the second processing module is used to read the to-be-identified from the storage module To identify the image frame to detect at least one target in the image frame; wherein, the second processing module sorts the image frames according to the image queue Read sequentially.

Compared with the prior art, the above-mentioned image recognition device and method use the first processing module to decode the video stream to obtain the image frame to be recognized, and use the multi-thread of the second processing module to recognize the image frame. Thereby improving the efficiency of image recognition and real-time target detection for each ultra-high-definition frame image.

10: The first processing module

20: Storage module

30: The second processing module

40: Communication bus

100: Image recognition device

P#1~P#N: Thread

PU#1~PU#N: processor unit

FIG. 1 is a schematic diagram of the structure of an image recognition device according to an embodiment of the present invention.

Fig. 2 is a functional block diagram of an image recognition device according to an embodiment of the present invention.

Fig. 3 is a flowchart of an image recognition method according to an embodiment of the present invention.

Please refer to FIG. 1, which is a schematic diagram of a preferred embodiment of the image recognition device of the present invention.

The image recognition device 100 includes a first processing module 10, a storage module 20, a second processing module 30 and a communication bus 40. The first processing module 10, the storage module 20, and the The second processing module 30 uses the communication bus 40 to complete mutual communication.

The storage module 20 may include at least one memory. The memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk drives, memory, plug-in hard disk drives, Smart Media Card (SMC), and secure Digital (Secure Digital, SD) card, flash memory card (Flash Card), at least one magnetic disk memory device, flash memory device, or other volatile solid-state memory device. The communication bus 40 includes, but is not limited to, a data bus, a power bus, a control bus, a status signal bus, and the like.

The first processing module 10 is used to obtain a video stream to be processed, and preprocess the video stream to obtain an image queue arranged in the order of frame playback.

In one embodiment, the first processing module 10 can obtain a video stream to be processed from a device with a video recording function, a video playback device, a storage device, etc. The preprocessing may include decoding of the video stream and frame image segmentation processing, and the frame image segmentation processing may divide a video into multiple image frames. It is understandable that the principle of video playback is to play image frames quickly and continuously. The first processing module 10 preferably processes the video stream into an image queue arranged in the order of frame playback. The initial image of the image queue is The image frame may be the first image of the video stream, and the end image frame of the image queue may be the last image of the video stream.

In order to facilitate subsequent identification of image frames, the first processing module 10 is also used to store the image queue in the storage module 20.

The video stream to be processed may be a video stream stored in the storage module 20, and the first processing module 10 may obtain the video stream to be processed from the storage module 20.

In one embodiment, the first processing module 10 may include one or more processors, and the processors may be a central processing unit (CPU), a microprocessor, or a digital signal processor (Digital Signal Processor). Processor, DSP), dedicated integrated circuit (Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. When the first processing module 10 includes multiple processors, the first processing module 10 may use multiple threads to preprocess the video stream to obtain image queues arranged in the order of frame playback.

The second processing module 30 is used to read the image frame to be identified from the storage module 20 and identify the image frame to detect at least one target in the image frame .

In one embodiment, the image frame to be recognized may be an image frame in the image queue for which no image recognition has been performed. The second processing module 30 may include at least one image recognition model to recognize the image frame. The image recognition model may be a recognition model trained by a machine learning algorithm, such as a deep residual network algorithm, a convolutional neural network algorithm, and the like.

The target can be set according to actual usage requirements. For example, the target can be a person, a bag (handbag, backpack), and the like. When the second processing module 30 needs to recognize multiple types of targets, each type of target can correspond to an image recognition model. For example, the second processing module 30 includes a first image recognition model and a second image recognition model. The first image recognition model is used to recognize a person in an image frame, and the second image recognition The model is used to identify the packets in the image frame, and the second processing module 30 can detect people and packets in the image frame.

In one embodiment, the second processing module 30 preferably reads and recognizes sequentially according to the sequence of the image frames of the image queue, that is, the image frames that are ranked first in the image queue are preferentially The second processing module 30 reads and recognizes.

In one embodiment, after the second processing module 30 reads the image frame from the storage module 20, it further adjusts the parameters of the image frame, and then adjusts the image after the parameter adjustment. Frame recognition, wherein the parameter adjustment may at least include image element adjustment and/or brightness adjustment, thereby improving the efficiency and accuracy of image recognition. The picture element adjustment may be, for example, to reduce the picture element of the image frame to reduce the amount of calculation.

In one embodiment, the second processing module 30 is also used for recognizing the The object category information of the target is stored in the image frame as the tag information of the target.

Please also refer to FIG. 2, the second processing module 30 may be a processor cluster, and the processor cluster supports hardware expansion. The processor cluster includes a plurality of threads P#1~P#N and a plurality of processor units PU#1~PU#N, where N is a natural number greater than 1. Each thread corresponds to each processor unit one by one. For example, the thread P#1 is used to read one frame of image to be recognized from the storage module 20 each time and transmit it to the processor unit PU#1, so as to analyze the at least one target object. Identify it. Each of the processor units PU#1~PU#N includes at least one image recognition model. The second processing module 30 can greatly improve the image recognition speed through the cooperation of the multi-thread P#1~P#N and the multi-processor unit PU#1~PU#N, and realize the processing of ultra-high-definition image frames. Instant target detection.

In one embodiment, due to the large amount of data in the ultra-high-definition image frame, the thread P#1 reads the image to be recognized one frame at a time from the storage module 20 to avoid reading the amount of data Too much affects the recognition speed, and then transmits it to the processor unit PU#1 for recognition. After the processor unit PU#1 recognizes one frame of image, the thread P#1 reads the next frame of the image to be recognized.

It is understandable that when the storage module 20 stores unidentified image frames, multiple threads P#1 to P#N will read the unidentified image frames in parallel. For example, the processor cluster includes eight threads P#1~P#8 and eight processor units PU#1~PU#8, and thread P#1 reads the first image in the image queue. The frame is sent to the processor unit PU#1 for identification, the thread P#2 reads the second image frame of the image queue and sends it to the processor unit PU#2 for identification, and the thread P#8 reads Take the eighth image frame of the image queue and send it to the processor unit PU#8 for identification. After the processor unit PU#1 completes the identification of the first image frame, the thread P#1 reads the ninth image frame of the image queue again and transmits it to the processor unit PU#1 for identification.

In one embodiment, the processor units PU#1~PU#N may be a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), and a dedicated integrated circuit (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

Fig. 3 is a flowchart of an image recognition method in an embodiment of the present invention. According to different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted.

In step S300, the first processing module 10 is used to obtain a video stream to be processed, and the video stream is preprocessed to obtain an image queue arranged in the order of frame playback.

Step S302, storing the image queue in the storage module 20.

Step S304: Use the second processing module 30 to read the image frame to be identified from the storage module 20, and identify the image frame to detect at least one of the image frames Target.

In one embodiment, the second processing module 30 preferably reads sequentially according to the sequence of the image frames of the image queue.

The above-mentioned image recognition device and method use the first processing module to decode the video stream to obtain the image frame to be recognized, and use the multiple threads of the second processing module to recognize the image frame, thereby improving the efficiency of image recognition. Realize real-time target detection for each ultra-high-definition frame image.

In summary, the present invention meets the requirements of an invention patent, and Yan filed a patent application in accordance with the law. However, the above are only the preferred embodiments of the present invention, and the scope of the present invention is not limited to the above embodiments. Anyone familiar with the art of the present case makes equivalent modifications or changes based on the spirit of the present invention. Should be covered in the scope of the following patent applications.

Claims (10)

An image recognition method applied to an image recognition device. The image recognition device includes a first processing module, a storage module, and a second processing module. The method includes: using the first processing module to obtain The video stream to be processed, and preprocess the video stream to obtain an image queue arranged in the order of frame playback; store the image queue in the storage module; and use the second processing module The group reads the image frame to be recognized from the storage module, and recognizes the image frame to detect at least one target in the image frame; wherein, the second processing module Read sequentially according to the sorting of the image frames of the image queue, the second processing module includes a plurality of threads and a plurality of processor units, and each thread corresponds to each of the processes one by one The step of using the second processing module to read the image frame to be recognized from the storage module and recognizing the image frame includes: controlling the thread every time from Reading a frame of the image to be recognized in the storage module and transmitting it to the processor unit; and using the processor unit to recognize the read frame of the image to be recognized. The method according to claim 1, wherein the step of using the first processing module to obtain a video stream to be processed includes: using the first processing module to read the to-be-processed video stream from the storage module The processed video stream. The method according to claim 1, wherein each of the processor units includes at least one image recognition model, and each of the image recognition models is used to recognize one type of target. The method according to claim 1, wherein the step of using the second processing module to recognize the image frame to detect at least one target in the image frame includes: using the first The second processing module recognizes the image frame to obtain object category information of the at least one target; and The object category information is stored in the image frame as tag information of the target object. The method according to claim 1, wherein the step of using the second processing module to identify the image frame includes: using the second processing module to adjust the parameters of the image frame, Then, the image frame after the parameter adjustment is recognized; wherein the parameter adjustment includes at least image element adjustment and/or brightness adjustment. An image recognition device includes a first processing module, a storage module, a second processing module, and a communication bus. The first processing module, the storage module, and the second processing module are The communication bus communicates with each other; the first processing module is used to obtain a video stream to be processed, and preprocess the video stream to obtain an image queue arranged in the order of frame playback; The first processing module is also used to store the image queue to the storage module; the second processing module is used to read the image frame to be identified from the storage module, and to The image frame is identified to detect at least one target in the image frame; wherein, the second processing module reads sequentially according to the sequence of the image frames in the image queue, the The second processing module includes a plurality of threads and a plurality of processor units, and each of the threads corresponds to each of the processor units one by one, and the threads are used to read from the storage module each time Take a frame of image to be recognized and send it to the processor unit. The device according to claim 6, wherein the first processing module is used to read the to-be-processed video stream from the storage module. The device according to claim 6, wherein each of the processor units includes an image recognition model to recognize the at least one target. The device according to claim 6, wherein the second processing module is further configured to store the recognized object type information of the target as the tag information of the target in the image frame. The device according to claim 6, wherein the second processing module is further configured to adjust the parameters of the image frame to be recognized, and to recognize the image frame after the parameter adjustment, wherein the parameter adjustment At least including image element adjustment and/or brightness adjustment.

Images