KR102286470B1 - LED Digital Signage Defect Detection system through Similarity Analysis of Original Image Information and Displayed Image Information - Google Patents

Abstract

An LED digital signage defect detection system includes a display part having a plurality of LEDs of forming a display area for displaying content; a camera for photographing the display part; and a control part that stores a content image to be displayed by a plurality of LED control signals of the plurality of LEDs, receives a photographed image from the camera, generates a histogram for the received photographed image and content image, and compares/analyzes the histogram of the display part to determine a defect area among the display areas of the display part.

Description

LED Digital Signage Defect Detection system through Similarity Analysis of Original Image Information and Displayed Image Information

The present invention relates to an LED digital signage failure detection system, and more particularly, to an LED digital signage failure detection system capable of detecting defects by imaging an LED digital signage and analyzing an image.

In general, the defect of the LED module of LED digital signage is measured by measuring the current input to and output from the LED module, and the amount of current consumed is measured to determine whether there is a short circuit or the LED is defective. Using this consumption current, it is possible to identify LED defects, but current measurement was not defective, but it could be defective in actual LED digital signage.

In addition, although a bad LED is detected by capturing an LED digital signage with a camera, there is a limit in identifying a bad LED because the bad LED is still identified only with the image captured by the camera.

Accordingly, it is an object of the present invention to provide an LED digital signage failure detection system capable of accurately identifying a defective area of a defective LED.

LED digital signage failure detection system according to the present invention for achieving the above object, a display unit having a plurality of LEDs forming a display area for displaying content; a camera for imaging the display unit; and storing the content image to be displayed by the plurality of LED control signals of the plurality of LEDs, receiving the captured image from the camera, generating a histogram for the received captured image and the content image, and the generated and a control unit that compares and analyzes the histogram of the captured image and the histogram of the content image to determine a defective area among the display areas of the display unit. By comparing the content image to be actually displayed on the LED digital signage with the captured image captured by the camera, the defective area can be identified and the distribution of the content image of the captured image can be precisely measured. there is.

Here, if the controller corrects the color of the received captured image to be within the color range of the stored content image, it is preferable to minimize the influence on the environment around the camera.

And the control unit, if the camera installation angle of the received captured image and the distortion of the lens of the camera is corrected, the captured image can be corrected to the same angle as the content image, so that comparison of images can be made easier.

Here, the control unit divides the corrected captured image and the content image into a plurality of regions, and compares the histogram of each of the divided regions of the corrected captured image and the content image to more precisely convert the captured image to the content image. Since the distribution can be precisely measured, the accuracy of identifying defects can be improved, which is preferable.

And the control unit divides one partition area determined to be defective among the plurality of partition areas into each of the LED modules formed by the plurality of LEDs, and each partition area of the corrected captured image and the content image. Comparing the histogram is preferable because it can focus and accurately identify areas suspected of being defective.

Here, the control unit may determine that the generation and supply of the plurality of LED control signals is defective when an area without change occurs among the display areas of the display unit when the screen is switched according to the plurality of LED control signals. .

And the control unit, when the brightness of one region of the captured image exceeds a predetermined value than the brightness according to the plurality of LED control signals, a power supply unit for supplying power and a plurality of LED modules formed by the plurality of LEDs It is preferable that it can be determined as at least one defect.

According to the present invention, since the defective area is identified by comparing the content image to be actually displayed on the LED digital signage with the captured image captured by the camera, it is possible to accurately identify the defective LED in which the defect has actually occurred.

1 is a flow block diagram of a method for determining a failure of an LED module using an LED digital signage failure detection system according to the present invention.
2 is an exemplary configuration diagram of an LED digital signage failure detection system.
3 is an exemplary view of the LED digital signage.
4 is an exemplary diagram of the content image of the LED digital signage.
5 is an exemplary view of the first inspection of the LED module.
6 is an exemplary view of the secondary inspection of the LED module.
7 to 9 are diagrams illustrating comparative analysis of histograms of captured images and content images.
10 is a control block diagram.

Hereinafter, the LED digital signage failure detection system 1 according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flow block diagram of a failure determination method of the LED modules 11 to n using the LED digital signage failure detection system 1 according to the present invention, and FIG. 2 is the LED digital signage failure detection system 1 It is an exemplary configuration diagram, FIG. 3 is an exemplary view of the display of LED digital signage, FIG. 4 is an exemplary diagram of the contents image of LED digital signage, and FIG. 5 is an exemplary view of the first inspection of the LED modules 11 to n. , FIG. 6 is an exemplary diagram of the secondary inspection of the LED modules 11 to n, FIGS. 7 to 9 are diagrams illustrating comparative analysis of histograms of captured images and content images, and FIG. 10 is a control block diagram.

LED digital signage failure detection system (1) includes a display unit (10), a power supply unit (20), a supply current sensing device (22), a current sensing unit (30), a power supply line (40), and a power control unit (42). ), a data supply line 44 , a power collection line 46 , a camera 50 , a camera driving unit 52 , a communication unit 60 , and a control unit 70 .

The display unit 10 constitutes an LED digital signage that displays content. The display unit 10 is composed of a plurality of LED modules 11 to n according to the size thereof, and each of the LED modules 11 to n is a plurality of LEDs 11-1 to 11-n arranged side by side in a line. made up of a bar shape. Each of the LED modules 11 to n is sequentially turned on instead of turned on at the same time because the power line is connected in series, but it is difficult to distinguish with the naked eye of use.

The LED modules 11 to n are 1/16Duty, 1/8Duty, 1/4Duty depending on the number of individual LED elements that can be turned on at one time. classified as static. Among them, in the case of 1/8Duty, only one of eight individual LED elements can be turned on at a time, so in the case of LED modules (11 to n) with different duties, the entire LED module (11 to It is impossible to measure the current consumed by each LED element in n) evenly.

So, when the ADDRESS that selects the displayed LED among the LED module control signals (DATA, CLOCK, LATCH, Out Enable, ADDRESS) and the corresponding Out Enable signal are detected and the current is measured, the total LED elements of the LED module (11 to n) are Current consumption can be measured. In addition, by detecting the current consumption when the Out Enable signal is in the off state of the screen, it is also possible to detect defects such as LED driver IC or LED element always on.

The power supply unit 20 is a switched mode power supply (SMPS) power supply and uses a switch control method for converting AC power supplied from the outside into DC power using a switching transistor or the like. The power supply unit 20 supplies power to each of the components of the LED digital signage failure detection system 1 . The power supply unit 20 may be provided in plurality to supply power to each of the LED modules 11 to n, respectively, and supply power to each of the plurality of LEDs 11-1 to 11-n of each of the LED modules 11 to n. An LED driver to supply may be provided.

The supply current sensing device 22 may be disposed at the rear end supplied to the components from the power supply unit 20 to detect the amount of supply current.

The current sensing unit 30 may be disposed at the front end of each of the LED modules 11 to n to sense the amount of supply current and the amount of consumed current. The current sensing unit 30 senses the amount of current drawn in and the amount of current drawn out to each of the LED modules 11 to n, and transmits it to the control unit 70 to be described later. The current sensing unit 30 may include a first current sensing unit 31 to an n-th current sensing unit n. The current sensing unit 30 applies an improved Kalman filter for current measurement of LED digital signage in order to minimize noise included in current measurement. The improved Kalman Filter applies the measured current value and the LED frequency value of the current screen as input as input factors, and reflects the characteristics of the LED modules (11 to n) that are repeatedly turned off and on, and the error for the rapidly changing current value reduce

The power supply line 40 is provided in plurality so as to supply current by being connected to all of the components including the LED modules 11 to n, which are components that consume power from the power supply unit 20 .

The power interrupter 42 may be disposed on the power supply line 40 between the plurality of LED modules 11 to n, and is disposed between the power supply 20 and the plurality of LED modules 11 to n to connect power. and can cut the current. The power interruption unit 42 may be provided with a selection valve capable of selectively supplying power.

The data supply line 44 is provided to connect the control unit 70, the power supply unit 20, and the plurality of LED modules 11 to n to transmit a module control signal. The data supply line 44 may be disposed together with the power supply line 40 or may be provided to transmit a module control signal using the power supply line 40 .

The power collection line 46 may be provided so as to collect the amount of current by connecting the control unit 70 from the inlet end and the outgoing end to the plurality of LED modules 11 to n.

The camera 50 may image the display unit 10 . The camera 50 may be supported on a support that supports the display unit 10 , and may be rotationally driven by the camera driving unit 52 . Accordingly, the display unit 10 can be imaged from different angles.

The communication unit 60 may transmit defect occurrence related information to an external central control room. The communication unit 60 may transmit data about the failure to an external central control room, and may receive data for remote control from the central control room.

The controller 70 stores the content image to be displayed by the plurality of LED control signals of the plurality of LEDs 11-1 to n, receives the captured image from the camera 50, and the received captured image and the stored content image is compared and analyzed to determine a defective area among the display areas of the display unit 10 .

The controller 70 may correct the color of the received captured image to be within the color range of the stored content image. For example, if the overall color atmosphere of the content image is green and the color atmosphere of the captured image is yellow, the calibration may be performed to obtain a green atmosphere by adding blue to the captured image. Thereby, color correction can be performed.

The control unit 70 corrects the distortion of the camera 50 installation angle and the lens of the camera 50 of the received captured image. Flattening correction is performed so as to be like a captured image captured in a state vertically arranged in front of the LED digital signage, and correction of an outer region due to a lens may be performed.

The controller 70 may generate a histogram of the captured image and the content image, and may compare the generated histogram of the captured image with the histogram of the content image. The histogram is a method of indicating the frequency, and in the case of colors, black/white/red/green/blue can be specified, and the frequency of colors for input data can be expressed. For example, the LEDs in some areas of the plurality of LEDs 11-1 to n may be shining too brightly or may be turned off because the LEDs are defective. Accordingly, the frequency of the color of the histogram of the captured image and the histogram of the content image will be different, and a defective LED in the defective area can be determined based on this.

The controller 70 may divide the corrected captured image and the content image into a plurality of regions, and may compare histograms of the corrected captured image and the content image with each of the segmented regions.

The control unit 70 divides one partition area determined to be defective among the plurality of partition areas into each of the LED modules 11 to n formed by the plurality of LEDs 11-1 to n, and the corrected captured image and the histogram of each partition area of the content image can be compared.

The control unit 70 may determine that the generation and supply of the plurality of LED control signals is defective when an area without change occurs among the display areas of the display when the screen is switched according to the plurality of LED control signals.

When the brightness of one area of the captured image exceeds a predetermined value than the brightness according to the plurality of LED control signals, the control unit 70 is configured by the power supply unit 20 for supplying power and the plurality of LEDs 11-1 to n. It may be determined that at least one of the formed LED modules 11 to n is defective.

1 is a flow block diagram of a failure determination method of LED modules 11 to n using the LED digital signage failure detection system 1 according to the present invention.

Before analyzing the histogram of the camera captured image, a virtual color range is designated based on the actual image to be displayed, and the input camera image is converted into a preset color range to minimize the influence on the camera environment.

Since there is distortion with respect to the installation angle and the lens in the camera image, the image is rectified after the camera calibration and the distortion correction process.

By comparing the difference and feature points of the histogram of the image being output from the display PC and the corrected image of the camera, the failure of the LED module at a specific location is determined.

When judging a faulty LED module, the inspection can be performed in two steps to increase the accuracy while using a small amount of resources.

(Stage 1)

1. When changing the screen from the current screen to the next screen, there should be a change, but there is no change.

1-1. Defective data cable in case of static module

1-2. In the case of large LED digital signage, the display local controller is defective.

2. Determine if the brightness of a specific part is too bright or too dark

2-1. Power supply, driver problem with LED module

3. Check whether one color is evenly distributed over the entire screen or a specific block

3-1. Error in video signal input/output part of display PC

3-2. over data cable

3-3. Module failure at the beginning of a specific color

4. Analyze the histogram of the entire screen and each section to check the similarity with the actual display and feature points

4-1. In case of possible defects in a specific module, proceed with two-step inspection

(Step 2)

1. During the first stage inspection, if the occurrence of a defective LED module is suspected, but an accurate judgment is not made due to camera flare and external illumination, etc., carry out if necessary.

2. Proceed if suspicion is determined during the first stage inspection

3. When switching from the current screen to the next screen, it is easy to check the defects of each module with the camera received image. Using the screen transition animation, the exact location of the defective module is determined by applying the screen transition animation step 1 inspection method.

(Compare histogram)

- Histogram calculation of the displayed original image

- Calculation of the histogram of the image displayed on the photographed and flattened LED digital signage

- The point with the highest similarity is calculated by shifting the bins of the histogram.

- Histogram similarity calculation

(Calculation of feature points)

- Extraction of feature points from the original image

- Extraction of feature points of the image displayed on the photographed and flattened LED digital signage

- Review whether there is a matching over a threshold value through feature point matching between two images

- Calculation of similarity based on feature points based on review values

(setting of threshold)

- Image histogram/feature point extraction based on camera images collected in various environments before shipment

- Finding the average/variance of image similarity (histogram/feature points) of modules without defects

- Set the distance of variance *n Gaussian as a threshold value based on the mean

(Determination of module failure)

The final similarity is calculated using the histogram and the feature point matching result.

Determination of module failure based on the calculated similarity

(Screen transition animation)

- LED digital signage screen usually consists of 16dot * 16dot standard LED module arrangement.

- When switching from one screen to the next, it is displayed for each LED module or based on the x-axis or y-axis, enabling more accurate histogram comparison and feature point calculation.

5 is an exemplary view of the first inspection of the LED module.

It checks the abnormality of the designated group, not the detailed inspection of the LED module unit (3X7).

6 is an exemplary view of the secondary inspection of the LED module.

If an abnormality is determined in the first inspection of the LED module, the secondary abnormal module inspection is designated by module unit (21X35) to determine the problematic LED module.

7 is a diagram illustrating comparative analysis of a histogram of a captured image and a content image.

7 (a) In a state in which the captured image (A) is divided, the captured image of the upper right region and the content image are compared. When the information that should be displayed does not come out in the first abnormal LED module check (a histogram is generated after processing a black and white image), or a histogram is created to know the information about color abnormality (a histogram is generated in RGB)

7 is an exemplary diagram of a histogram of a content image after processing a black-and-white image.

7 (a) A histogram of the content image in the upper right area is generated in a state in which the content image (A) is partitioned.

7 (b) A1 is a content image, and A2 is an image processing result of the content image. Image processing is processing for displaying an image in black or white.

8 is an exemplary diagram illustrating a histogram of a captured image.

8 (a) A histogram of the upper right area is generated in a state in which the captured image A is divided.

8 (b) A1 is a captured image, and A2 is an image processing result of the captured image. Image processing is processing for displaying an image in black or white.

9 is an exemplary view of generating a histogram of a captured image.

9 (a) A histogram of the upper right area is generated in a state in which the captured image A is divided.

9 (b) A1 is a captured image, and A2 is an image processing result of the captured image. Image processing is processing for displaying an image in black or white.

FIG. 7 is a content image, and it can be confirmed that there is an abnormality in the LED module through a histogram comparison with FIGS. 8 and 9, which are captured images. 7, 8 and 9 (c) are the histogram generation result of A2, and since the background color is black, the intensity of black is strong in the histogram. 255 of does not exist. Through the histogram of (c) of the content image of FIG. 7, it can be confirmed that the text of the current content image exists. Through the histogram of FIG. (8) and the histogram of FIG. (9), it can be confirmed whether the LED module operates normally or has an abnormality. Through the histogram of FIG. 8(c), it can be confirmed that 255 white is present because letters exist, and it can be confirmed that the LED module is operating normally through comparison with the histogram of FIG. 7(c). In FIG. 9(c) , it can be confirmed that there is no white 255 because there are no characters, and the abnormality of the LED module can be determined by comparing it with the histogram of FIG. 7(c).

Modifiable embodiments other than the above embodiments will be described.

The controller 70 may determine the defective display area of the defective LED among the plurality of LEDs 11-1 to n and change the content so that the content can be displayed in the normal display area except for the defective display area.

The control unit 70 stores the normal current range for the plurality of LED control signals of the plurality of LEDs 11-1 to n, and current values for the plurality of LED control signals from the plurality of LEDs 11-1 to n. LEDs outside the normal current range can be judged as defective. The power supply unit 20 including a current sensing function is used as a power supply device for driving the LED digital signage LED modules 11 to n, and the actual consumption current of the LED modules 11 to n is the LED module control signal and It is collected through the interlocked current sensing unit 30 . The measured current measured in this way is interlocked with the LED display control unit 70 and analyzed together with the current display contents and the set luminance value to determine whether there is an abnormality in the LED digital signage.

In this way, since it is possible to measure the current consumed by the power supply unit 20 and the current consumed by the LED modules 11 to n, the amount of current leaked from the power supply line and the amount of current consumed by the LED modules 11 to n Since it is possible to measure and determine whether the current consumption is appropriate, it is possible to detect a bad location of the LED modules 11 to n and the risk of a short circuit of the LED digital signage.

The controller 70 may determine a defective display area by comparing the content to be displayed by the plurality of LED control signals of the plurality of LEDs 11 to n with the image captured by the camera 50 .

The controller 70 may change the ratio of at least some of the content areas.

The controller 70 may delete at least some of the content areas.

The controller 70 may change the position of at least some of the content areas.

The control unit 70 may stop supplying power to the defective LED in the defective display area.

The normal display area may include a content display area for displaying contents of the content and a base area other than the content display area.

The control unit 70 may stop supplying power to the plurality of LEDs in the base area.

The controller 70 may change the color of the content display area when it is determined that the color is defective in all of the plurality of LEDs 11 to n.

The control unit 70 may transmit the generation of the bad LED to the central system.

It may further include an LED module support or an LED support for supporting the LED modules 11 to n and the LEDs 11-1 to n, and the LED modules 11 to n and the LEDs 11-1 to n. may be fixedly installed on the LED module support or the LED support, but may be installed detachably for the convenience of maintenance. Accordingly, the defective LED modules 11 to n and the LEDs 11-1 to n can be easily replaced.

Due to the above-described LED digital signage failure detection system 1, the defective area is identified by comparing the content image to be actually displayed on the LED digital signage with the captured image captured by the camera 50. can be accurately identified.

If the color of the received captured image is corrected to be within the color range of the stored content image, the influence on the environment around the camera 50 can be minimized. If the camera 50 installation angle and the distortion of the lens of the camera 50 of the received captured image are corrected, the captured image can be corrected at the same angle as the content image, thereby making it easier to compare images.

By creating a histogram for the captured image and the content image, and comparing the histogram of the captured image with the histogram of the content image, the distribution of the content image can be precisely measured for the captured image, so the accuracy of identifying defects can be improved. .

If the corrected captured image and the content image are divided into a plurality of regions, and the histogram of each of the divided regions of the corrected captured image and the content image is compared, it is possible to accurately measure the distribution of the captured image and the content image. Accuracy of figuring out can be improved.

One partition area determined to be defective among the plurality of partition areas is divided into LED modules 11 to n formed by a plurality of LEDs 11-1 to n, respectively, and the corrected captured image and content image are partitioned. By comparing the histograms of each region, it is possible to focus and accurately identify regions suspected of being defective. When a screen is switched according to a plurality of LED control signals, when an area without change occurs among the display areas of the display unit, it may be determined that the generation and supply of the plurality of LED control signals are defective. When the brightness of one area of the captured image exceeds a predetermined value than the brightness according to the plurality of LED control signals, the plurality of LEDs formed by the power supply unit 20 for supplying power and the plurality of LEDs 11-1 to n It is preferable because it can be determined that at least one of the modules 11 to n is defective.

1: LED digital signage failure detection system 10: display unit
11 to n: 1st LED module to nth LED module
11-1 to n: 1st LED to nth LED
20: power supply 22: supply current sensing device
30: current sensing unit
31 to n: first current sensing unit to nth current sensing unit
40: supply power line 42: power interruption unit
44: data supply line 46: power collection line
50: camera 52: camera driving unit
60: communication unit 70: control unit

Claims (7)

In the LED digital signage failure detection system,
a display unit having a plurality of LEDs forming a display area for displaying content;
a camera for imaging the display unit; and
store the content image to be displayed by a plurality of LED control signals of the plurality of LEDs, receive a captured image from the camera, generate a histogram for the received captured image and the content image, and generate the captured image and a control unit that compares and analyzes the histogram of the image and the histogram of the content image to determine a defective area among the display areas of the display unit,
The control unit is
Among the display areas of the display unit, a normal display area includes a content display area in which content is displayed and a base area other than the content display area, and the defective display area of the defective LED among the plurality of LEDs is identified and the content is displayed in the defective display area. to be displayed in the content display area of the normal display area except for, perform at least one of changing the ratio of the content, deleting the content, and changing the location of the content, and the defective LED in the defective display area and the An LED digital signage failure detection system for stopping power supply to the plurality of LEDs in the base area and changing the color of the content display area when it is determined that the color is defective in all of the plurality of LEDs in the normal display area.
According to claim 1,
The control unit is
An LED digital signage failure detection system that corrects the color of the received captured image to be within the color range of the stored content image.
According to claim 1,
The control unit is
An LED digital signage failure detection system for correcting distortion of the camera installation angle and the lens of the camera of the received captured image.
4. The method of claim 3,
The control unit is
An LED digital signage failure detection system that divides the corrected captured image and the content image into a plurality of regions, and compares histograms of the corrected captured image and the segmented region of the content image.
5. The method of claim 4,
The control unit is
An LED that divides one partition area determined to be defective among the plurality of partition areas into each of the LED modules formed by the plurality of LEDs, and compares the histogram of each partition area of the corrected captured image and the content image Digital signage failure detection system.
According to claim 1,
The control unit is
LED digital signage failure detection system for judging that the generation and supply of the plurality of LED control signals is defective when an area without change occurs in the display area of the display unit when the screen is switched according to the plurality of LED control signals.
According to claim 1,
The control unit is
When the brightness of one area of the captured image exceeds a predetermined value than the brightness according to the plurality of LED control signals, at least one of a power supply unit for supplying power and a plurality of LED modules formed by the plurality of LEDs is defective. LED digital signage failure detection system.

Images