TWI718442B - Convolutional neutral networks identification efficiency increasing method and related convolutional neutral networks identification efficiency increasing device - Google Patents

Abstract

A convolutional neutral networks identification efficiency increasing method is applied to a related device. The convolutional neutral networks identification efficiency increasing method includes analyzing an input image to acquire foreground information, utilizing the foreground information to generate a foreground mask, and transforming the input image into an output image via the foreground mask. The output image is used to be an input of the convolutional neutral networks identification for preferred object identification efficiency.

Description

Neural network-like identification performance improvement method and neural network-like identification performance improvement device Set

The present invention provides an image recognition method and device, in particular a neural network-like recognition performance improvement method and a neural network-like recognition performance improvement device applied to image recognition.

Traditional image recognition technology based on neural network calculations directly uses raw surveillance images as input information. The large amount of information contained in the original surveillance image greatly limits the improvement of image recognition performance; even if a small range of specific images are selected from the original surveillance image for image recognition, an attempt is made to reduce the amount of information to improve computing performance. The test object will still be affected by the surrounding complex environment background, and the required identification result cannot be obtained quickly and accurately. Therefore, how to design a method that helps to improve the performance of neural network identification is one of the key development topics of the related surveillance industry.

The present invention provides a neural network-like recognition performance improvement method and a neural network-like recognition performance improvement device applied to image recognition to solve the above-mentioned problems.

The scope of patent application of the present invention discloses a method for improving the performance of neural network recognition, which includes analyzing an input image to obtain foreground information, using the foreground information to generate a foreground mask, and passing the input image through the foreground mask Convert to an output image. The output image is used as the imported data for neural network identification to improve the performance of object identification.

The patent application scope of the present invention also discloses a neural network-like recognition performance improvement device, which includes an image generator and an arithmetic processor. The image generator is used to obtain an input image. The arithmetic processor is electrically connected to the image generator. The arithmetic processor is used to analyze the input image to obtain foreground information, use the foreground information to generate a foreground mask, and convert the input image into an output image through the foreground mask, which can effectively improve the complexity of neural network algorithms. Object recognition performance in the environment. The output image is used as the imported data for neural network recognition to improve the performance of object recognition.

The neural network recognition performance improvement method and device of the present invention first separate foreground information from the input image, and define the foreground mask according to the pixel value distribution of the foreground information. The input image can be effectively filtered out by the conversion of the foreground mask. Necessary information, and the generated output information can be used as the imported data for neural network identification to improve the accuracy of neural network identification. The input image is not limited to color modes such as RGB, YUV, HSL or HSV. The input image and the converted foreground information, the foreground mask, and the output image belong to the calculation of each pixel value, so they all have substantially the same image size. In addition, the pixel grayscale value of the output image can be selectively limited to a specific range, the purpose is to reduce the storage capacity required by the neural network-like recognition performance improvement device, and more effectively process large-capacity image information.

10: Neural network identification performance improvement device

12: Image generator

14: arithmetic processor

I: Monitoring screen

I1: Input image

I2: Prospect Information

I3: Foreground mask

I4: Output image

H: Histogram of foreground information

H1: The first straight square model

H2: The second histogram model

S1: First group

S2: The second group

S200, S202, S204, S206, S208, S210: steps

S700, S702, S704, S706, S708, S710, S712, S714, S716: steps

Figure 1 is a functional block diagram of a neural network recognition performance improvement device according to an embodiment of the present invention.

FIG. 2 is a flowchart of a method for improving the recognition performance of a neural network according to an embodiment of the present invention.

Figures 3 to 6 are schematic diagrams of the input image in different conversion stages according to the embodiment of the present invention.

Figure 7 is a flow chart of generating a foreground mask according to an embodiment of the present invention.

FIG. 8 is a schematic diagram of a histogram of foreground information according to an embodiment of the present invention.

FIG. 9 is a schematic diagram of the pixel distribution type used to analyze the foreground mask according to an embodiment of the present invention.

Please refer to FIG. 1. FIG. 1 is a functional block diagram of a neural network recognition performance improvement device 10 according to an embodiment of the present invention. The neural network-like recognition performance improvement device 10 may include an image generator 12 and a computing processor 14 electrically connected together. The image generator 12 is used to obtain the input image I1. The image generator 12 can be an image capture device, which directly obtains the image information of the monitoring range as the input image I1; or, the image generator 12 can also be an image receiver, which receives images generated by an external image capture device in a wired or wireless manner The information is treated as input image I1. The input image I1 is mainly used in the object recognition technology based on neural network (Convolutional Neutral Networks, CNN); therefore, the computing processor 14 executes a set of neural network identification performance enhancement methods, which can effectively improve the neural network-like Object recognition performance of road algorithm in complex environment.

Please refer to Figures 2 to 6. Figure 2 is a flowchart of a neural network recognition performance improvement method according to an embodiment of the present invention. Figures 3 to 6 are respectively different input images I1 according to an embodiment of the present invention. Schematic diagram of the conversion phase. The neural network identification performance improvement method described in FIG. 2 is applicable to the neural network identification performance improvement device 10 shown in FIG. 1. First, perform step S200 and step S202 to obtain a monitoring frame I related to the monitoring range, and use the object detection technology to select the range of the input image I1 on the monitoring frame I. The implementation mode shown in Fig. 3 is to select a small range of input image I1 in the monitoring frame I, but the actual application is not limited to this; for example, the entire frame of the monitoring frame I can be regarded as the input image I1. Then, step S204 and step S206 are executed to generate the background information of the input image I1, and the difference between the input image I1 and the background information is calculated to obtain Get Prospect Information I2. The input image I1 can be used to create background information through a Gaussian mixture model (Mixture of Gaussians, MOG) or background subtraction based on a neural network algorithm, or obtain background information by any other algorithm.

Steps S204 and S206 analyze the input image I1 to obtain foreground information I2. The foregoing first obtaining of background information and then calculating the difference between the input image I1 and the background information is only one of the multiple methods of obtaining the foreground information I2, and the actual application is not limited to this. Next, step S208 and step S210 are performed to generate a foreground mask I3 using the foreground information I2, and then the input image I1 is converted into an output image I4 by the foreground mask I3. If the monitoring screen I is captured from a complex environment, such as a busy road or an intersection with people and vehicles, even if a small range of input image I1 is drawn from the monitoring screen I, the input image I1 will still cover many backgrounds that affect the accuracy of detection pattern. The present invention filters out the background objects of the input image I1 through the foreground information I2. As shown in Figure 6, the background objects of the output image I4 have been erased. Therefore, the output image I4 is used as the imported data for neural network identification, which can reduce complexity The interference of background objects in the environment effectively improves object recognition performance and detection accuracy.

Please refer to FIGS. 3 to 8. FIG. 7 is a flowchart of generating a foreground mask I3 according to an embodiment of the present invention, and FIG. 8 is a schematic diagram of a histogram H converted from foreground information I2 according to an embodiment of the present invention. First, perform steps S700 and S702 to calculate the histogram H of the foreground information I2, and divide the histogram H into a plurality of groups according to the pixel value range; for example, it is divided into at least the first group S1 and the second group S2. The pixel value range of the group S1 is smaller than the pixel value range of the second group S2. Step S704 is executed again to compare the number of pixels in the second group S2 with a predetermined parameter. The predetermined parameter may be determined according to statistical data, for example, determined according to the environment of the monitoring screen I, or the ratio of the number of pixels between the second group S2 and the first group S1 is set to define the predetermined parameter. The number of pixels in the second group S2 is greater than the predetermined parameter, indicating that there are dynamic objects in the input image I1; the number of pixels in the second group S2 is less than the predetermined parameter, and it is not yet certain whether the objects in the input image I1 remain static or interfered by noise.

If the number of pixels in the second group S2 is greater than the predetermined parameter, it indicates that there is a significant change between the input image I1 and the background information, and step S706 is executed to set the foreground threshold; for example, the foreground threshold may be the percentage of the average value of all pixels in the histogram H Of forty. The percentage of the prospect threshold is not limited to this value, it depends on the design requirements. Then, step S708 is executed, the pixels in the foreground information I2 whose pixel values are higher than the foreground threshold are classified as the first group of pixels, and the pixels below the foreground threshold are classified as the second group of pixels. Next, step S710 is performed to set the pixel values of the first group of pixels and the second group of pixels in the foreground mask I3 to the first value and the second value, respectively, to generate the foreground mask I3. For example, the first value can be 1, as shown in Fig. 5, the grid-free area of foreground mask I3, and the second value can be 0, as shown in Fig. 5, the grid of foreground mask I3 area.

If the number of pixels in the second group S2 is less than the predetermined parameter, there is little change between the input image I1 and the background information, and step S712 is executed to determine whether the first group S1 meets a specific condition. The specific condition means that the first group S1 has a larger number of pixels, and the actual number depends on the environment and statistical data. If the first group S1 meets certain conditions, it means that the pixel distribution in the histogram H is concentrated in the low-end range, and the object in the input image I1 is deemed to remain stationary. Step S714 is executed, and the pixel values of all pixels in the foreground mask I3 are set to the first Numerical value; when the first value is 1, the input image I1 can be directly used as the output image I4, which is used as the imported data for neural network identification. If the first group S1 does not meet the specific conditions, it means that the pixel distribution in the histogram H is scattered, which means that the input image I1 is interfered by noise. Step S716 is executed, and the pixel values of all pixels in the foreground mask I3 are set to the second value; When the second value is 0, the input image I1 is directly discarded.

In step S210, the input image I1 is converted to the output image I4 through the foreground mask I3, and the product of all the pixel values of the input image I1 and the corresponding pixel value of the foreground mask I3 can be directly calculated. The resulting product is the output image I4 Each pixel value; or, after calculating the product of all the pixel values of the input image I1 and the corresponding pixel value of the foreground mask I3, you can further filter out the position corresponding to the foreground from these products The first set of products of pixel positions that do not belong to the second value and the second set of products of pixel positions that belong to the second value in the mask I3. The second set of products can be classified as background. If it is set to the second value, the background pixels belonging to the second set of products in the output image I4 are black, which may affect the color rendering of the objects in the output image I4, so you can change The second set of products is replaced with a reference value (as shown in the single slanted area of the output image I4 as shown in FIG. 6), and the first set of products and the reference values are combined as the pixel values of the output image I4. For example, the color of the object under test (such as pedestrian) in the output image I4 is generally black and white information. If the second set of products is set as the second value (black), it is easy to be confused with the pattern of the object under test. The two sets of products can optionally be set to other colors, such as gray, to clearly separate the object to be tested and the background.

Please refer to FIGS. 7-9. FIG. 9 is a schematic diagram of the pixel distribution type used to analyze the foreground mask according to an embodiment of the present invention. Step S704 is to compare the number of pixels of the second group S2 with a predetermined parameter. The present invention can further preset the first and second histogram models H1 and H2 as shown in FIG. 9. If the histogram H of the foreground information I2 is similar to the first rectangular model H1, which means that the number of pixels in the second group S2 is larger (greater than the predetermined parameter), step S708 can be continued; if the number of pixels in the second group S2 is too small ( Less than a predetermined parameter), step S712 is further executed to determine whether the histogram H of the foreground information I2 is similar to the second histogram model H2. If the histogram H is similar to the second histogram model H2, it means that it meets the specific condition that the first group S1 has a larger number of pixels, and step S714 is performed to generate the relevant foreground mask I3; if the histogram H is different from the second histogram model H2, it means That is, the number of pixels in the first group S1 is small and does not meet the specific conditions, and step S716 is executed to discard the input image I1. The first histogram model H1 can present a visualized graphic style with predetermined parameters, and the second histogram model H2 can present a visualized graphic style with specific conditions, but the actual style is not limited to the foregoing embodiment.

In summary, the neural network-like recognition performance improvement method and device of the present invention first separate the foreground information from the input image, and classify the foreground information according to the pixel value distribution of the foreground information to define the foreground mask in different situations. The conversion of the image by the foreground mask can effectively filter out unnecessary information, so Generate output information as the imported data for neural network identification, which can improve the accuracy of neural network identification. In particular, the input image is not limited to color modes such as RGB, YUV, HSL, or HSV. The input image and the converted foreground information, the foreground mask, and the output image belong to the arithmetic conversion of each pixel value, so they all have substantially the same image size. In addition, the pixel grayscale value of the output image can be selectively limited to the range of 0~128, the purpose is to reduce the storage capacity required by the neural network recognition performance improvement device, and more effectively process large-capacity image information; therefore, the foreground is blocked. The mask is a binary image, and the output image is a 256-level or 128-level grayscale image. Compared with the prior art, the present invention filters out the background noise of the input image, which helps to improve the performance of neural network recognition.

The foregoing descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made in accordance with the scope of the patent application of the present invention should fall within the scope of the present invention.

S200, S202, S204, S206, S208, S210: steps

Claims (12)

A kind of neural network recognition performance improvement method, which includes: analyzing an input image to obtain foreground information; calculating a histogram of the foreground information; the histogram is divided into at least a first group and a first group according to the pixel value range Two groups, and a pixel value range of the first group is smaller than a pixel value range of the second group; comparing the number of pixels in the second group with a predetermined parameter; the number of pixels in the second group is greater than the predetermined parameter , Set a foreground threshold; pixels in the foreground information whose pixel values are higher than the foreground threshold are classified as a first group of pixels; pixels in the foreground information whose pixel values are lower than the foreground threshold are classified as a second group of pixels Group of pixels; the pixel value corresponding to the first group of pixels in a foreground mask is set to a first value, and the pixel value corresponding to the second group of pixels is set to a second value; and the input image passes through the The foreground mask is converted into an output image, where the output image is used as the imported data for neural network identification to improve the performance of object identification. The neural network recognition performance improvement method as described in claim 1 further includes: using object detection technology to select a range of the input image in a monitoring screen. The neural network recognition performance improvement method as described in claim 1, which limits the pixel grayscale value of the output image to the range of 0-128. A kind of neural network recognition performance improvement method, which includes: analyzing an input image to obtain foreground information; Calculating a histogram of the foreground information; the histogram is at least divided into a first group and a second group according to pixel value ranges, and a pixel value range of the first group is smaller than a pixel value range of the second group; Compare the number of pixels in the second group with a predetermined parameter; when the number of pixels in the second group is less than the predetermined parameter, determine whether the first group meets a specific condition; if the first group meets the specific condition, the The pixel values of all pixels in the foreground mask are set to a first value to generate a foreground mask; and the input image is converted into an output image through the foreground mask, wherein the output image is used as the imported data for neural network identification, To improve the performance of object recognition. The neural network recognition performance improvement method as described in claim 4, wherein when the first group does not meet the specific condition, the pixel values of all pixels in the foreground mask are set to the second value. A kind of neural network recognition performance improvement method, which includes: analyzing an input image to obtain foreground information; calculating a histogram of the foreground information; the histogram is divided into at least a first group and a first group according to the pixel value range Two groups, and a pixel value range of the first group is smaller than a pixel value range of the second group; comparing the number of pixels in the second group with a predetermined parameter; generating a foreground mask according to the comparison result; and calculating the The product of all the pixel values of the input image and the corresponding pixel value of the foreground mask is used to generate an output image, where the output image is used as the imported data for neural network recognition to improve the performance of object recognition. The neural network recognition performance improvement method as described in claim 6 further includes: using object detection technology to select a range of the input image in a monitoring screen. The neural network recognition performance improvement method as described in claim 6, wherein the input image is converted to the output image through the foreground mask includes: calculating the correspondence between all pixel values of the input image and the foreground mask respectively The product of pixel values; a first set of products whose positions correspond to pixel positions in the foreground mask that do not belong to a second value in the foreground mask are filtered out from the products; the positions corresponding to the foreground are filtered out from the products A second set of products of pixel positions belonging to a second value in the mask is replaced, and the second set of products is replaced with a reference value; and the output image is generated using the first set of products and the reference values. The neural network recognition performance improvement method as described in claim 6, which limits the pixel grayscale value of the output image to a range of 0-128. A kind of neural network recognition performance improvement device, which includes: an image generator for obtaining an input image; and an arithmetic processor electrically connected to the image generator, and the arithmetic processor is used for executing the request items 1 to The neural network recognition performance improvement method described in one of claim 3 or a combination thereof can effectively improve the object recognition performance of the neural network algorithm in a complex environment. A kind of neural network recognition performance improvement device, which includes: an image generator for obtaining an input image; and An arithmetic processor electrically connected to the image generator, and the arithmetic processor is used to execute the neural network identification performance improvement method as described in one of request item 4 to request item 5 or a combination thereof, which can effectively improve the neural network The object recognition performance of network algorithms in complex environments. A kind of neural network recognition performance improvement device, which includes: an image generator for obtaining an input image; and an arithmetic processor electrically connected to the image generator, and the arithmetic processor is used for executing the request items 6 to The neural network recognition performance improvement method described in one of claim 9 or a combination thereof can effectively improve the object recognition performance of the neural network algorithm in a complex environment.

Images