TWI831688B - Method for monitoring brightness chances in images and device thereof - Google Patents

Abstract

A method for monitoring brightness changes in images and a device thereof are provided. The method includes: capturing and obtaining a plurality of first images from a capture area by an image capture device; generating an ignore mask according to the first images; generating a grayscale average image according to the first images, and masking the grayscale average image by the ignore mask to obtain a reference image; capturing a second image from the capturing area by the image capturing element; masking the second image by the ignore mask and subtracting grayscale values of each pixel position in the reference image from the second image masked by the ignore mask to obtain an image to be tested; and, judging whether brightness of the second image changes according to the image to be tested.

Description

Method and device for monitoring brightness changes in images

The present invention relates to an image recognition technology used in industrial automation, and in particular, to a method and device for monitoring brightness changes in an image.

In industrial automation (Automation) technology, automated equipment machines often use image capture technology to perform operations such as object detection, feature point capture in images, workpiece alignment, etc. However, image capture will be affected by the shooting environment and Affected by the parameters used in image capture. In other words, maintaining the stability of the shooting environment is an important factor for the application of image capture technology in industrial automation technology.

Currently, image analysis monitoring combined with person identification can be used to confirm the stability of the shooting environment. For example, parameters such as brightness or sharpness can be monitored in key areas of the captured image, and personnel can be asked to confirm to ensure that the aforementioned environment has not been adjusted due to external factors (for example, manual adjustment/camera aperture setting, etc.) ) and variation, it may be possible to avoid areas where some materials or backgrounds have brightness/sharpness variations under normal circumstances through human identification, ensuring that the image values in the obtained areas are not affected by the aforementioned factors and have sufficient reference value. However, there are often errors in the way people are identified. Mistakes often occur, causing the accuracy of the aforementioned image analysis and monitoring to be questioned.

The present invention provides a method and device for monitoring brightness changes in an image, which can automatically ignore areas with no reference value or with lower reference value in the image, and use areas with reference value in the image to judge the image. Whether the brightness is changed to save labor and reduce the chance of human error in industrial automation technology.

A method for monitoring brightness changes in an image according to an embodiment of the present invention includes the following steps: capturing and obtaining a plurality of first images from a capture area through an image capture device; according to the multiple first images Generate an ignoring mask; generate a grayscale average image based on the plurality of first images, and mask the grayscale average image with the ignoring mask to obtain a reference image; capture the image through The capturing device captures a second image from the capture area; masks the second image by using the ignore mask, and subtracts each element in the reference image from the masked second image. The grayscale value of the pixel position is used to obtain the image to be measured; and it is determined whether the brightness in the second image changes based on the image to be measured.

A device for monitoring brightness changes in an image according to an embodiment of the present invention includes an image capturing device and a processor. The image capturing device is used to capture a plurality of first images and second images from the capturing area. The processor is coupled to the image capturing device. The processor is configured to: capture and obtain a plurality of first images from the capture area via the image capture device, generate an ignore mask based on the plurality of first images, and generate an ignore mask based on the plurality of first images. The image produces a grayscale average image, and the grayscale average is masked by the ignore mask. average the images to obtain a reference image, capture a second image from the capture area via the image capture device, mask the second image by the ignore mask, and store the masked image The gray scale value of each pixel position in the reference image is subtracted from the second image to obtain the image to be tested, and whether the brightness in the second image changes is determined based on the image to be tested.

Based on the above, the method and device for monitoring brightness changes in images described in embodiments of the present invention use images obtained by the image capture device to automatically calculate and generate image masks that have no reference value or have low reference value. Mask is used as an ignore mask to remove areas in the image that may misjudge the brightness of the image. Then, this ignore mask is used to mask other images and serve as a reference for brightness changes of the image, so as to achieve nearly fully automated monitoring of image brightness, thereby saving manpower and reducing the chance of human error.

100: Device for monitoring brightness changes in images

105:Vehicle

107:Object

109: Capture area

110:Image capture device

120: Processor

130:Memory

S210~S270, S410~S445: Each step of the method of monitoring brightness changes in images

310: First image

315:Ignore mask

317: Grayscale average image

319:Reference image

320: Second image

325: Image to be tested

510:Difference image

520: First brightness mask

530: Second brightness mask

540: Measurement does not include images

545: Average image of multiple images without measurement

X: Grayscale reference value

FIG. 1 is a schematic diagram of a device 100 for monitoring brightness changes in an image according to an embodiment of the present invention.

FIG. 2 is a flowchart of a method 200 for monitoring brightness changes in an image according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of images in method 200.

FIG. 4 is a detailed flow chart of step S220 in FIG. 2 .

Figure 5 is a schematic diagram of each image in Figure 4.

Figure 6 shows each grayscale value (presented on the X-axis) in the measurement-excluding image 540. A schematic diagram showing the number of pixels (presented on the Y-axis) as a histogram.

FIG. 1 is a schematic diagram of a device 100 for monitoring brightness changes in an image according to an embodiment of the present invention. The device 100 for monitoring brightness changes in an image includes an image capturing device 110 and a processor 120 . The image capture device 110 is used to capture images from the capture area 109 . Specifically, as shown in Figure 1, the industrial automation equipment is equipped with a carrier 105, and an object 107 (for example, a PCB board) is placed above the carrier 105. The capture area 109 includes a part of the carrier 105 and a part of the object 107 .

The image capture device 110 captures images from the capture area 109. In this embodiment, these images can be used not only for monitoring brightness changes in the image, but also for object detection and feature point capture in the image. Images used in image processing technology such as picking, workpiece alignment, etc. in industrial automation.

The processor 120 is coupled to and controls the image capturing device 110 . The memory 130 is coupled to the processor 120 to temporarily store images and corresponding data being processed. The processor 120 is used to execute a method for monitoring brightness changes in an image according to the embodiment of the present invention.

Under normal circumstances, there may be brightness/sharpness variations or defective areas due to different product carriers, materials or backgrounds, and these areas in the image are difficult to use as a reference for judging changes in image brightness. In addition, some areas in the capture area 109 are already saturated in brightness. If the brightness is further increased, these areas will still be difficult to change significantly. Similarly, the brightness of some areas in the capture area 109 is already too dark or extremely low. If the brightness is further reduced, these areas will still be difficult to change significantly.

Therefore, the device and method for monitoring brightness changes in images described in this embodiment generate corresponding ignored masks by comparing pixel positions with large grayscale differences between the collected images, thereby utilizing the ignored masks to remove possible As the area of image brightness misjudgment, the image masked by ignoring the mask is compared with the average brightness of the normal image, thereby achieving the function of automatically monitoring whether the image brightness changes.

FIG. 2 is a flowchart of a method 200 for monitoring brightness changes in an image according to an embodiment of the present invention. FIG. 3 is a schematic diagram of images in method 200. FIG. 2 can be applied to the device 100 for monitoring brightness changes in the image of FIG. 1 , and the method 200 is mainly implemented through the processor 120 of FIG. 1 .

Please refer to FIGS. 2 and 3 simultaneously. In step S210 , the processor 120 of FIG. 1 captures and obtains a plurality of first images 310 from the capture area 109 through the image capture device 110 of FIG. 1 . The plurality of first images 310 described in this embodiment and the second image 320 described below are all captured by the image capture device 110 in Figure 1. For the convenience of subsequent explanation, these images are named with different names. mark.

In step S220, the processor 120 of FIG. 1 calculates and generates the ignoring mask 315 based on the first images 310. In this embodiment, two first images 310 that are captured at a similar time are used as the basic images used to generate the ignore mask 315 in this embodiment of the present invention. However, any image captured by the image capture device 110 in FIG. 1 can be used as the basic image for generating the ignore mask 315 according to the embodiment of the present invention, and is not limited thereto. How to calculate and generate the ignore mask 315 using the two first images 310 captured sequentially will be described in detail in subsequent embodiments (eg, FIG. 4 and FIG. 5 and corresponding descriptions).

In step S230, the processor 120 of FIG. 1 uses these first images 310 to A grayscale average image 317 is calculated and generated, and the grayscale average image 317 is masked by ignoring the mask 315 to obtain a reference image 319 . In this embodiment, the grayscale values of each pixel position in the collected first images 310 are averaged to generate the grayscale average image 317 , and the aforementioned ignore mask 315 is used to generate the grayscale average image 317 Areas with low reference value are masked, thereby generating a reference image 319 . The reference image 319 mainly records the pixel positions and their grayscale values in the grayscale average image 317 that are not obscured by the ignored mask 315 . These pixel positions that are not obscured by the ignored mask 315 are of high reference value for determining whether the brightness in the image changes. In this embodiment, the first image 310 is used to first organize the pixel positions and their grayscale values with higher reference values into the reference image 319, which can be used when subsequently determining whether the brightness in the image has changed.

So far, this embodiment uses the ignore mask 315 of step S220 and the reference image 319 of step S230 to process other images (such as the second image 320) in steps S240 to S270, thereby determining the second Whether the brightness in image 320 changes.

Specifically, in step S240 , the processor 120 of FIG. 1 captures the second image 320 from the capture area 109 of FIG. 1 via the image capture device 110 of FIG. 1 . In step S250, the processor 120 of FIG. 1 masks the second image 320 by using the ignore mask 315 to ignore the areas with low reference value. In step S260, the processor 120 of FIG. 1 subtracts the grayscale value of each pixel position in the reference image 319 from the masked second image 320 to obtain an absolute value between the second image 320 and the reference image 319. The difference image is the image to be tested 325. In step S270, the processor 120 of FIG. 1 determines whether the brightness in the second image 320 changes according to the image 325 to be tested.

The detailed steps of step S260 are: the processor calculates the average grayscale value of each pixel in the image 325 to be measured. It should be noted that some pixel positions of the image to be measured 325 are obscured, so the grayscale values corresponding to these pixel positions are not calculated in the aforementioned average grayscale value. Next, the processor determines whether the average grayscale value exceeds a brightness threshold. The brightness threshold can be set by the user of the embodiment of the present invention, for example, "10". When the average grayscale value exceeds the brightness threshold "10", the processor will perform a prompt operation, thereby using this prompt operation to present the information that "the brightness of the image to be measured has been changed" to the corresponding person. The brightness threshold can be a single value or a range of values.

The detailed steps of generating the ignore mask 315 in step S220 of FIG. 2 will be described in detail here. FIG. 4 is a detailed flow chart of step S220 in FIG. 2 . Figure 5 is a schematic diagram of each image in Figure 4.

Please refer to both Figure 4 and Figure 5. In step S410, the processor obtains the difference image 510 between the two first images 310 captured sequentially. In detail, the processor subtracts the grayscale values corresponding to each pixel position in the two first images 310 to generate the difference image 510 .

Steps S420 and S430 of this embodiment are to capture extremely dark areas (for example, a set of pixel locations with grayscale values less than a first critical grayscale value (for example, "25")) and extremely bright areas in the first image 310 Areas (for example, a set of pixel locations with grayscale values greater than the second critical grayscale value (for example, "230")), these two areas with low reference values serve as brightness masks to exclude pixels with low reference values. Those who apply this embodiment can adjust the values of the first critical gray scale value and the second critical gray scale value according to their needs. For example, the first critical gray scale value can also be "10", and the first critical gray scale value can also be " 245』.

Specifically, here, taking the first grayscale value as 25 and the second grayscale value as 230 as an example, in step S420, the processor compares the grayscale of each pixel position in one of the first images 310. The level value and the first critical gray level value are used to capture the area of the first image 310 that is smaller than the first critical gray level value, thereby obtaining the first brightness mask 520 . Accordingly, the grayscale value of each pixel in the first image 310 that is obscured by the first brightness mask 520 is less than the first critical grayscale value. In step S430, the processor compares the grayscale value of each pixel position of one of the first images 310 with the second critical grayscale value to capture that the first image 310 is greater than the second critical grayscale value. area, and then obtain the second brightness mask 530. Accordingly, the gray level value of each pixel in the first image 310 that is blocked by the second brightness mask 530 is greater than the second critical gray level value. In this embodiment, the image on the left side of the two first images 310 in Figure 5 is regarded as "one of the first images 310". Those who apply this embodiment can use any of the first images 310 according to their needs. An image is used as "one of the first images 310" described in steps S410 to S440 in this embodiment.

In step S440, the processor calculates and obtains the ignore mask 315 based on the difference image 510 of step S410, the first brightness mask 520 of step S420, and the second brightness mask 530 of step S430. In the embodiment of the present invention, the simplest way to generate the ignore mask 315 is to add the difference image 510, the first brightness mask 520 and the second brightness mask 530 to generate a rough ignore mask 315. In another embodiment of the present invention, the ignore mask 315 can be generated by step S441 to step S445 in FIG. 4 .

In step S441 , the processor adds the grayscale values of each pixel position in the difference image 510 , the first brightness mask 520 and the second brightness mask 530 to generate a measurement-free image 540 .

In step S442, the processor calculates a grayscale reference value X based on the grayscale value of each pixel in the measured image 540. For details, please refer to Figure 6. Figure 6 is a schematic diagram showing the number of pixels (presented on the Y-axis) of each grayscale value (presented on the X-axis) in the measured image 540 as a histogram, that is, , presenting the number of pixels corresponding to each grayscale value as a histogram. The smaller the grayscale value in Figure 6 is, it means that these pixel positions have high reference value and small variation for image brightness. Therefore, in this embodiment of the present invention, counting starts from the grayscale value 0 and accumulates to a certain number of pixels. This number of pixels can be 30% of the total number of pixels in the image and corresponds to the grayscale reference value X calculated in step S442. In other words, the sum of the number of pixels from grayscale value 0 to grayscale reference value 30% of the areas have reference value. 30% of the total number of pixels is a brightness reference area. Depending on the shooting situation, if the brightness stable area is known to be extremely small, the percentage may need to be adjusted downwards. However, if the general stable area is greater than 30%, 30% of the total number of pixels can be used as the reference area. Among them, the above-mentioned "approximately equal to" may be completely equal, or may not be completely equal (that is, within a tolerance range, for example, within plus or minus 10%).

Returning to FIGS. 4 and 5 , in step S443 , the processor compares the grayscale values of each pixel position in the unmeasured image 540 based on the grayscale reference value X in step S442 . In step S444, when the grayscale value of each pixel position in the measurement ignore image 540 is greater than the grayscale reference value Hood 315. A grayscale value greater than the grayscale reference value The degree of detection can be ignored. Therefore, steps S441 to S444 generate the ignore mask 315. In this embodiment, in order to avoid factors such as image drift or errors, in step S445, the processor will appropriately expand each area in the ignored mask 315 by an expanded pixel value (for example, moderately expand "5" pixels) , thereby producing the dilated ignore mask 315.

In this embodiment, in steps S441 to S445 of FIG. 4 and FIG. 5 , the two first images 310 are acquired sequentially through the difference image 510 , the first brightness mask 520 , and the second brightness mask 530 . To generate the measurement-excluding images 540, those who apply this embodiment can also generate and collect a large number of measurement-excluding images 540 from any two of the plurality of first images 310, and perform these measurement-excluding images 540. Average to generate an average image 545 of multiple omission images, and use this average image 545 to generate and adjust the ignore mask 315 in the manner of the aforementioned step S444.

Determining whether the brightness in the second image changes in the image to be tested includes the following steps. First, calculate the average gray scale value of each pixel in the image to be measured, and then determine whether the average gray scale value exceeds the brightness threshold. When the average grayscale value exceeds the brightness threshold, a prompt operation is performed, and the prompt operation is used to present that the brightness of the image to be measured has been changed.

In summary, the method and device for monitoring brightness changes in images described in embodiments of the present invention use images obtained by the image capture device to automatically calculate and generate images that have no reference value or have low reference value. Image mask is used as an ignore mask to remove areas of the image that may misjudge the brightness of the image. Then, use this ignore mask to mask other images and use it as a reference for the brightness change of the image to achieve close range. It fully automatically monitors image brightness to save manpower and reduce the chance of human error.

S210~S270: Each step of the method for monitoring brightness changes in images

Claims (10)

A method for monitoring changes in brightness in an image, including: capturing and obtaining a plurality of first images from a capture area through an image capture device; generating an ignore mask based on the plurality of first images; A grayscale average image is generated based on the plurality of first images, and the grayscale average image is masked by the ignoring mask to obtain a reference image; and the image capturing device is used to obtain a grayscale average image from all the grayscale average images. The capture area captures a second image; the second image is masked by the ignore mask, and the masked second image is subtracted from the reference image at each pixel position. The grayscale value is used to obtain an image to be measured; and it is determined whether a brightness in the second image changes according to the image to be measured. According to the method of claim 1, generating the ignoring mask based on the plurality of first images includes: obtaining a difference image between two of the plurality of first images; comparing The grayscale value of each pixel position of one of the plurality of first images and a first critical grayscale value are used to obtain a first brightness mask, wherein the first image is covered by the first The grayscale value of each pixel blocked by the brightness mask is less than the first critical grayscale value; comparing the grayscale value of each pixel position in one of the plurality of first images with a second A critical grayscale value is obtained to obtain a second brightness mask, wherein the grayscale value of each pixel of the first image obscured by the second brightness mask is greater than the second critical grayscale value. boundary gray level value, and the first critical gray level value is less than the second critical gray level value; and obtaining the said difference image, the first brightness mask and the second brightness mask according to the difference image, the first brightness mask and the second brightness mask Masks are ignored. According to the method of claim 2, obtaining the ignore mask based on the difference image, the first brightness mask and the second brightness mask includes: combining the difference image, the first brightness mask The grayscale values of each pixel position in the brightness mask and the second brightness mask are added together to generate a measurement-free image; a grayscale is calculated based on the grayscale value of each pixel in the measurement-free image. Reference value; compare the gray scale value of each pixel position in the measurement-excluding image based on the gray-scale reference value; and when the gray-scale value of each pixel position in the measurement-excluding image is greater than the gray level When the reference value is used, the corresponding pixel position is recorded, and the ignore mask is generated based on the recorded pixel position. The method according to claim 1, wherein the plurality of first images and the second images include a part of the object. According to the method of claim 1, determining whether the brightness in the second image has changed based on the image to be tested includes: calculating an average grayscale value of each pixel in the image to be tested; determining Whether the average gray scale value exceeds a brightness threshold; and when the average gray scale value exceeds the brightness threshold, a prompt operation is performed, The prompt operation is used to present that the brightness in the second image has been changed. A device for monitoring brightness changes in an image, including: an image capture device for capturing a plurality of first images and a second image from a capture area; and a processor coupled to the An image capture device, wherein the processor is configured to: capture and obtain the plurality of first images from the capture area via the image capture device, according to the plurality of first images An ignore mask is generated, a grayscale average image is generated based on the plurality of first images, and the grayscale average image is masked by the ignore mask to obtain a reference image. The image capture device captures the second image from the capture area, masks the second image by the ignore mask, and subtracts the masked second image from the masked second image. Referring to the grayscale value of each pixel position in the image to obtain an image to be tested, and judging whether a brightness in the second image changes based on the image to be tested. The device of claim 6, wherein the processor is further configured to: obtain a difference image between two of the plurality of first images, compare the plurality of first images The grayscale value of each pixel position of one of the images and a first critical grayscale value are used to obtain a first brightness mask, wherein each part of the first image obscured by the first brightness mask The grayscale value of pixels is smaller than the first adjacent boundary grayscale value, comparing the grayscale value of each pixel position of one of the plurality of first images with a second critical grayscale value to obtain a second brightness mask, wherein the first The grayscale value of each pixel of the image obscured by the second brightness mask is greater than the second critical grayscale value, and the first critical grayscale value is less than the second critical grayscale value, and , the ignore mask is obtained based on the difference image, the first brightness mask and the second brightness mask. The device of claim 7, wherein the processor is further configured to: compare the grayscale values of each pixel position in the difference image, the first brightness mask and the second brightness mask. Summing up to generate a measurement-excluding image, calculating a gray-scale reference value based on the gray-scale value of each pixel in the measurement-excluding image, and comparing the values in the measurement-excluding image based on the gray-scale reference value The grayscale value of each pixel position, and when the grayscale value of each pixel position in the measured image is greater than the grayscale reference value, record the corresponding pixel position, and based on the recorded The pixel position that produces the ignore mask. The device of claim 6, wherein the plurality of first images and the second images include a part of the object. The device of claim 6, wherein the processor is further configured to: calculate an average grayscale value of each pixel in the image to be measured; determine whether the average grayscale value exceeds a brightness threshold; And when the average grayscale value exceeds the brightness threshold, a prompt operation is performed, and the prompt operation is used to present that the brightness in the second image has been changed.