KR102486562B1 - image analysis system, method, computer program and computer readable recording medium to analyze whether electronic sign board is malfunctioning - Google Patents

Abstract

An image analysis system for determining whether an electric sign board is malfunctioning is disclosed. The system includes a data processing part that compares the histogram of a reference image and the histogram of a target image to determine whether the electric sign board is malfunctioning. Therefore, it is possible to determine whether the electric sign board is malfunctioning through image histogram comparison.

Description

Image analysis system, method, computer program and computer readable recording medium to analyze whether electronic sign board is malfunctioning}

The present invention relates to an image analysis system, method, computer program and computer readable recording medium for analyzing whether an electric sign is malfunctioning.

An electric sign is installed on the road to provide various information to drivers, pedestrians, etc. located on the road.

Electric signage is a device to improve the efficiency of traffic flow and the safety of passage by providing road users with real-time information on traffic, roads, weather conditions, and control due to construction. Arranged in a dot matrix form and selectively turns on specific light emitting devices to express traffic-related information including various texts or images.

That is, the electric signs receive traffic information from the traffic information center and provide traffic information such as road traffic conditions, construction information, unexpected situations, and detours to drivers in real time.

In this way, the electric signs display various traffic information in characters or figures, and it is necessary to monitor the electric signs to check whether the contents transmitted from the traffic information center are properly displayed on the electric signs.

Conventional electric sign monitoring methods include a dot monitoring method using a semiconductor and a monitoring method using a camera.

Here, the dot monitoring method using a semiconductor has a disadvantage in that the circuit configuration of the electric sign increases because a current or voltage detection device must be installed to check whether each light emitting element operates normally, and it is necessary to check the actually displayed characters. There was a downside to not being able to.

On the other hand, the monitoring method using a camera is a method in which the manager directly compares and checks the real-time captured video of the electric sign with his eyes. This method requires a lot of time and concentration because the manager has to directly compare the inputted traffic information and the output screen. The downside was that it was necessary.

The present invention has been made to solve the above problems, and an object of the present invention is to provide an image analysis system, method, computer program and computer readable recording medium for determining whether an electric sign is malfunctioning through image histogram comparison. .

An image analysis system according to an embodiment of the present invention for achieving the above object includes a data processing unit that compares a histogram of a reference image and a histogram of an image to be judged to determine whether an electric sign is malfunctioning.

In addition, an image analysis method according to an embodiment of the present invention for achieving the above object includes comparing a histogram of a reference image and a histogram of an image to be judged to determine whether an electric sign is malfunctioning.

In addition, the computer readable recording medium according to an embodiment of the present invention for achieving the above object may store a program for performing the above-described image analysis method.

In addition, a computer program stored in a computer-readable recording medium according to an embodiment of the present invention for achieving the above object may include a program code for executing the above-described image analysis method.

According to various embodiments of the present invention, it is possible to determine whether the electric sign is malfunctioning by comparing histograms for each of the reference image and the image to be judged.

In addition, according to the present invention, by comparing the histogram of each of the reference image and the image to be judged and using it as a basis for determining whether the electric sign is malfunctioning, it is possible to automate monitoring of malfunction of the electric sign using a camera.

In addition, according to the present invention, it is possible to more accurately determine whether an electric sign is malfunctioning by dividing a reference image and an image to be judged into predetermined regions and comparing histograms for each of the divided predetermined regions.

In addition, the present invention can more accurately determine whether the electric sign is malfunctioning by using environmental information around the electric sign.

In addition, according to the present invention, it is possible to determine the location of a malfunctioning area within the electric sign.

In addition, according to the present invention, the image analysis system can provide various traffic-related services by collecting traffic information of vehicles and pedestrians, traffic law violation information, and crime prevention information.

1 is a conceptual diagram illustrating an image analysis system according to an embodiment of the present invention.
2 is a block diagram showing the configuration of an image analysis system according to an embodiment of the present invention.
3 is a flowchart illustrating a method for determining whether an electric sign malfunctions in a data processing unit according to a first embodiment of the present invention.
4 is an exemplary diagram illustrating a reference image and an image to be judged and histograms thereof according to the first embodiment of the present invention.
5 is an exemplary diagram illustrating an electric sign indicating a variable lane situation according to an embodiment of the present invention.
6 is a flowchart illustrating a method for determining whether an electric sign malfunctions in a data processing unit according to a second embodiment of the present invention.
7 is an exemplary diagram in which a reference image and an image to be judged are divided into predetermined regions according to an embodiment of the present invention.
8 is an exemplary diagram illustrating a reference image and an image to be judged and histograms thereof according to a second embodiment of the present invention.
9 is a flowchart illustrating a method for determining whether an electric sign malfunctions in a data processing unit according to a third embodiment of the present invention.
10 and 11 are diagrams illustrating a histogram conversion method according to a third embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram illustrating an image analysis system according to an embodiment of the present invention. Referring to FIG. 1 , an image analysis system 1000 may include a photographing device 100 and a data processing unit 330 . In addition, the image analysis system 1000 may further include an operating device (not shown). For example, the operating device may be provided in a control center.

The photographing device 100 is a device for photographing an image to be judged, and the photographing device 100 may be installed in a position where the electric sign 11 may be photographed and may photograph an image. Here, the image captured by the photographing device 100 may be used to determine whether the electric sign 11 is malfunctioning. For example, the image to be determined captured by the photographing device 100 may include an electric sign displaying traffic-related information. Here, the traffic-related information may include a variety of traffic-related information, such as surrounding weather conditions, road construction, congestion conditions, accident conditions, major news information, and the like.

In addition, to capture not only the electric sign 11 but also surrounding conditions (eg, road conditions, etc.), the photographing device 100 is an omnidirectional camera capable of covering all directions of the road or surrounding roads. It may be composed of a plurality of cameras having an angle of view that

An image to be determined captured by the photographing device 100 may be transmitted to the data processing unit 330 . In this case, the image to be determined captured by the photographing device 100 may be transmitted to the data processing unit 330 in real time or at regular intervals.

The data processing unit 330 may determine whether the electric sign 11 is malfunctioning by comparing the histogram of the image to be judged obtained from the photographing device 100 with the histogram of the pre-stored reference image.

Also, the data processing unit 330 may be provided in a site where the photographing device 100 is installed (eg, a traffic controller enclosure) or an operating device installed in a control center.

Specifically, the operating device may perform a control function based on an analysis result of whether the data processing unit 330 is malfunctioning. For example, the operating device controls the output of the electric sign 11 based on the analysis result of the data processing unit 330 (eg, controls to display traffic-related information by excluding malfunctioning parts) or communicates with the vehicle. It is possible to provide traffic-related information expressed on the electric sign 11 or a notification allowing a user (eg, a control agent) to check whether the electric sign 11 is malfunctioning in real time. Here, the notification enabling confirmation of malfunction may be implemented as an alarm notifying that a malfunction diagnosis is required.

Meanwhile, the data processing unit 330 may be implemented as a separate module from the photographing device 100 and the operating device. For example, the data processing unit 330 may be implemented as a separate infrastructure equipped with a vehicle-to-infrastructure (V2I) module installed in one area of a road. In this case, when the electric sign 11 malfunctions, the data processing unit 330 may transmit information about whether the electric sign 11 malfunctions to the vehicle through vehicle-to-infrastructure (V2I) communication. In addition, the data processing unit 330 may transmit traffic-related information displayed on the electric sign 11 to the vehicle through vehicle-to-infrastructure (V2I) communication.

As such, the image analysis system 1000 according to an embodiment disclosed in this document can quickly and easily detect various errors that may occur in an electric sign by analyzing an image to be determined obtained from the photographing device 100. .

The configuration and operation of the image analysis system 100 will be described in more detail below.

2 is a block diagram showing the configuration of an image analysis system according to an embodiment of the present invention. Referring to FIG. 2 , the image analysis system 1000 may include all or part of a photographing unit 310 , a sensor unit 320 , a data processing unit 330 , and a database unit 340 .

The photographing unit 310 may perform substantially the same function as the photographing device 100 described with reference to FIG. 1 .

The sensor unit 320 detects an object such as a vehicle or a person and tracks the movement of the object, thereby measuring a location or moving distance of the object (eg, a radar (radio detection and ranging, RADAR), a lidar ( At least one of a light detection and ranging (LIDAR), an image sensor, and a laser detection and ranging (LADAR). According to embodiments, the image analysis system 300 may use the photographing unit 310 and the sensor unit 320 cooperatively. For example, the image analysis system 300 may preferentially use the sensor unit 320 in order to more accurately detect the moving position, distance, direction, or speed of the object, and when it is necessary to identify the shape of the object. In this case, the photographing unit 310 may be used. According to the embodiment, the photographing unit 310 and the sensor unit 320 may be implemented as one integrated module.

Also, the sensor unit 320 may include an environment sensor capable of detecting photographing environment information of the electric sign 11 . Here, the environment sensor may include at least one of a temperature sensor, a humidity sensor, an anemometer, and an illumination sensor.

The database unit 340 may store various programs and data necessary for the operation of the image analysis system 1000 . In particular, the database unit 340 may classify and store the reference image and histogram previously calculated for the reference image according to a predetermined condition. Here, the predetermined condition may include at least one of content and environment information of the base image.

In addition, the database unit 340 may classify and store images to be judged received from each of the plurality of photographing devices 100 installed in a plurality of regions according to identification information of the photographing device.

In addition, the database unit 340 may classify and store the received image to be determined according to the predetermined conditions described above, for example, the content and environment information of the image to be determined.

That is, the database unit 340 may map and store predetermined conditions to images as shown in the table below.

video contents environmental information 1st video speed warning Time: 10-11 am Weather conditions: rain 2nd video safe driving Time: 1-3 pm Weather conditions: Snow

Here, the database unit 340 has been described as being included in the image analysis system 1000 as an example, but is not limited thereto. According to another embodiment of the present invention, the database unit 340 can be managed by a company that manages images, and the video analysis system 1000 accesses the database unit 340 that performs a master database function, , It may be implemented to download and use various data from the database unit 340. For example, the database unit 340 may be managed by an image company that stores, processes, and manages the above-described reference image.

Meanwhile, the data processing unit 330 may acquire various information through at least one of the photographing unit 310 and the sensor unit 320 . For example, the data processing unit 330 may obtain traffic information, and the traffic information may include a road image. In addition, the traffic information includes vehicle image, number of vehicles, vehicle driving speed, vehicle driving direction, vehicle driving lane, vehicle moving distance, vehicle traffic volume, vehicle waiting length, vehicle occupancy rate, pedestrian image, and pedestrian traffic information. It may further include at least one of number, gait speed, or gait direction. In addition, the traffic information may include identification information including at least one of the type of vehicle, color of vehicle, size of vehicle, driving lane, vehicle number, or appearance of a pedestrian. Additionally, the data processing unit 330 may obtain crime prevention information through the photographing unit 310, and the 'security information' may include at least one of life crime prevention information and traffic crime prevention information.

The data processing unit 330 may detect objects and/or events using traffic information (eg, images). Objects may include vehicles, people (eg, vehicle occupants, pedestrians, etc.), animals, and the like. An 'event' is a dangerous situation that has occurred on the road and may mean a state that needs to be notified to vehicles or pedestrians. For example, the data processing unit 330 detects event types such as vehicle stop, vehicle accident, reverse driving, pedestrian/wild animal appearance, falling object, roadkill, landslide, fog, or pothole, and the location where the corresponding event occurred. can do. Information indicating the type of event or location of the event may be referred to as 'event information'.

The data processing unit 330 may detect an event through data obtained from at least one sensor in addition to traffic information. For example, the data processing unit 330 may include an acceleration sensor for detecting an impact on the road, a positioning sensor for detecting a landslide on a slope adjacent to the road, a temperature/humidity sensor for detecting weather changes such as temperature/humidity, and a load Event information may be obtained based on data received from at least one of a motion sensor for detecting a kill and an acoustic sensor.

Meanwhile, the analysis result data (eg, at least one of traffic information, crime prevention information, event information, malfunction determination result information, and malfunction area location information) of the data processing unit 330 may be transmitted to a communication relay or a vehicle or operating device. .

Meanwhile, the data processing unit 330 may determine whether the electric sign 11 is malfunctioning by comparing the histogram of the reference image and the histogram of the image to be judged.

The data processing unit 330 may be provided in a site where the photographing device 100 is installed (eg, a traffic controller box) or an operating device installed in a control center.

For example, when the data processing unit 330 is provided at a site where the photographing device 100 is installed, the data processing unit 330 obtains an image to be determined captured by the photographing device 100, and obtains an image to be judged from the database unit 340. At least one of a base image matching a predetermined condition with the judgment target image and a pre-calculated histogram of the base image may be received through a communication network. In addition, the data processing unit 330 may determine whether the electric sign 11 is malfunctioning by comparing a histogram of the image to be determined captured by the photographing device 100 with a histogram of the reference image. Here, the predetermined condition may include at least one of content and environment information of the base image.

As another example, when the data processing unit 330 is provided in an operating device installed in a control center where the photographing device 100 is installed, the data processing unit 330 receives the image to be determined captured by the photographing device 100 through a communication network, and , The database unit 340 may detect a base image matching a predetermined condition with the received image to be judged and a pre-calculated histogram for the base image. In addition, the data processing unit 330 may determine whether the electric sign 11 is malfunctioning by comparing the histogram of the reference image and the histogram of the image to be judged.

On the other hand, in the above example, the image analysis system 1000 has been described as an example of determining whether the electric sign 11 is malfunctioning using an image, but the implementation example of the present invention is not limited thereto. That is, the video analysis system 1000 of the present invention is applied to all environments requiring real-time determination of whether or not a display is malfunctioning (eg, baseball field lighting signs, soccer field lighting signs, outdoor lighting signs of buildings, etc.) to determine malfunctions. can be used to do

Hereinafter, for convenience of description, the image analysis system 1000 according to an embodiment of the present invention will be described as being applied to a driving environment on a road of a vehicle as an example.

Hereinafter, an image analysis system according to various implementation examples of the present invention will be described in more detail with reference to the drawings and FIGS. 3 and 4 .

Hereinafter, with reference to FIGS. 3 to 9 , a method for determining whether an electric sign is malfunctioning by the data processing unit 330 will be described in more detail.

[Method for determining whether an electric sign is malfunctioning according to the first embodiment]

3 is a flowchart illustrating a process of determining whether an electric sign malfunctions in a data processing unit according to an embodiment of the present invention.

Referring to FIG. 3 , the data processing unit 330 may obtain an image to be judged (S110). Here, the image to be judged is an image to be compared to determine whether the electric sign 11 is malfunctioning, and the image to be judged may be a real-time captured image of the electric sign 11 using the photographing device 100 .

In addition, the data processing unit 330 may perform a pre-processing process on the image to be judged. Specifically, the data processing unit 330 distinguishes between the electric sign 11 area to be determined and an area other than the electric sign 11 (eg, a background) in the image to be judged, and other than the electric sign 11 A preprocessing process of removing the region of may be performed. In this case, the pre-processing process may be omitted according to the state of the image to be determined.

Also, the data processing unit 330 may obtain a base image from the database unit 340 (S120). Here, the reference image is an image that is a comparison standard for determining whether the electric sign 11 is malfunctioning, and the reference image may be a photographed image of the electric sign 11 in a situation where the electric sign 11 is operating normally. there is.

In this case, both the reference image and the image to be judged analyzed and processed by the data processing unit 330 may be images in a situation in which traffic-related information input to each other is the same. For example, the reference image is an image taken in a situation where the normally operating electric sign 11 displays traffic-related information of “A”, and the image to be judged is the electric sign 11 is the traffic-related information of “A”. It may be an image captured in real time in a situation of displaying.

Meanwhile, the data processing unit 330 may use at least one of content and environment information of the image to be determined in order to detect a base image to be compared with the image to be determined from among a plurality of base images in order to acquire the base image. Specifically, the data processing unit 330 may detect at least one of content and environment information of the image to be determined, and detect a base image matching at least one of the detected content and environment information. For example, the data processing unit 330 photographs an electric sign such as weather information (fine dust, fog, heavy rain, heavy snow, humidity and temperature, etc.), season information, time information (sunrise, noon, sunset, night time, time zone, etc.) It is possible to determine environmental information that affects the processing, and to detect a reference image of an environment identical to or similar to the environment information of the image to be judged from among a plurality of pre-stored reference images.

Meanwhile, the data processing unit 330 may perform pre-processing on the acquired base image.

Next, the data processing unit 330 may calculate a histogram for the obtained image to be judged (S130).

Also, the data processing unit 330 may calculate a histogram for the obtained reference image (S140). Here, the histogram of the base image may be a pre-calculated and stored value for each of the base images stored in the database unit 340 . That is, the data processing unit 330 may calculate a histogram of the obtained reference image by detecting a pre-calculated and stored histogram of the reference image.

Alternatively, the data processing unit 330 may calculate a histogram in real time from the acquired reference image.

Here, the histogram is a graph representing the distribution of pixel values of an image, where the X-axis represents pixel values and the Y-axis represents the number of pixel values.

Specifically, the data processing unit 330 may calculate the number of pixels corresponding to pixel values for each color channel for each of the reference image and the image to be judged, thereby calculating a histogram. Here, the color channel may include at least one of a red channel, a green channel, a blue channel, and gray.

The data processing unit 330 may determine a different color channel for calculating the histogram according to the color configuration of the display of the electric sign 11 or the photographing device 100 that captures the electric sign 11 .

On the other hand, according to an embodiment of the present invention, for real-time processing of data, the data processing unit 330 is a histogram for the reference image (here, the histogram is a method for determining whether an electric sign is malfunctioning according to various embodiments of the present invention. (including histograms or modified histograms divided into predetermined unit areas) are calculated in advance, and at least one of environment information at the time when the reference image was captured and contents corresponding to the reference image are mapped to the database unit 340. It can be sorted and stored.

Meanwhile, the data processing unit 330 may calculate a similarity by comparing the histogram of the image to be judged with the histogram of the reference image (S150). In this case, the histogram may be a histogram obtained by combining at least one color channel. For example, the histogram may be a histogram in which R (Red), G (Green), and B (Blue) channels are combined.

Specifically, the data processing unit 330 may calculate the degree of similarity by overall comparing each graph of the histogram of the image to be judged and the histogram of the reference image. Alternatively, the data processing unit 330 extracts a plurality of points corresponding to portions of the graph corresponding to the histogram of the image to be determined and the histogram of the reference image, respectively, and compares patterns of lines connecting the plurality of extracted points to determine similarity. can be calculated This will be described with additional reference to FIG. 4 .

4 is an exemplary diagram illustrating a reference image, an image to be judged, and each histogram according to an embodiment of the present invention.

Referring to FIG. 4, FIG. 4(a) shows a reference image of a situation in which the electric sign 11 operates normally, and FIG. 4(d) shows traffic in a partial area 21 of the electric sign due to malfunction of the electric sign. It shows an image to be judged for which related information is not displayed. In this case, the data processing unit 330 may calculate a histogram for the reference image as shown in FIG. 4(b) and a histogram for the image to be determined as shown in FIG. 4(e). As shown in FIGS. 4(b) and 4(e), a difference 22 occurs between the histogram of the reference image and the histogram of the reference image due to the influence of the malfunction area 21.

In this case, the data processing unit 330 may calculate the degree of similarity by overall comparing each graph of the histogram of the image to be judged and the histogram of the reference image. At this time, the data processing unit 330 may compare the area of the graph to calculate the degree of similarity.

In addition, the data processing unit 330 selects points 23 and 24 from the histogram of the reference image shown in FIG. 4(c) and the histogram of the image to be determined shown in FIG. 4(f) according to a preset extraction criterion. After extracting and connecting the extracted points for each image with adjacent points, the similarity can be calculated by comparing patterns of the connected points. Here, the extraction criterion may be variably determined according to at least one of the state and environment information of the reference image or the image to be judged. For example, the extraction criterion may be extracted at predetermined intervals. Here, the state of the image may include resolution, quality, and the like of the image.

That is, the data processing unit 330 may compare the pattern of the points 23 extracted from the histogram of the reference image with the pattern of the points 24 extracted from the histogram of the image to be judged to calculate the degree of similarity.

Then, the data processing unit 330 may determine whether the electric sign is malfunctioning (S160). Specifically, the data processing unit 330 may determine whether or not a malfunction occurs depending on whether the calculated similarity exceeds a threshold value.

For example, if the calculated similarity is greater than the threshold, the data processing unit 330 determines that the electric sign 11 operates normally, and if the calculated similarity is smaller than the threshold, the electric sign 11 malfunctions. can be judged to be In this case, the threshold may be variably determined according to environmental information.

That is, the method for determining whether the electric sign is malfunctioning according to the first embodiment of the present invention is a method for determining whether the electric sign 11 is malfunctioning through a histogram comparison of each of the entire reference image and the entire subject image. In the method for determining whether the electric sign is malfunctioning according to the first embodiment, the normal operation of the electric sign 11 is ON or the electric sign 11 is currently OFF (or the opposite situation), Useful for detecting a situation in which the amount of traffic-related information displayed on the electric sign 11 is not large (for example, an electric display indicating only a variable lane situation as shown in FIG. 5) or a situation in which a lot of errors occur in the electric sign 11 It can be.

[Method for Determining Malfunction of Electric Signs According to the Second Embodiment]

6 is a flowchart illustrating a method for determining whether an electric sign malfunctions in a data processing unit according to a second embodiment of the present invention. Referring to FIG. 6 , the method for determining whether an electric sign is malfunctioning according to the second embodiment of the present invention further includes steps S230 and S240, and is different from the method for determining whether an electric sign is malfunctioning according to the first embodiment of the present invention. there is

Specifically, the data processor 330 may divide each of the obtained image to be determined and the reference image into predetermined regions (S230 and S240), and calculate a histogram for each of the divided regions (S250 and S260). Here, the criterion for dividing the predetermined region may be determined according to text information and graphic information of traffic-related information included in each of the obtained image to be determined and the reference image.

Here, in text information, a character may be a concept including all special characters such as letters such as Korean or English, numbers, and symbols. Such character information includes character type information (for example, letters, numbers, or special characters), character color information, character size information, character spacing information (for example, space between characters, space between lines, etc.), and character position information. and character structure information (eg, word, morpheme, syllable unit, etc.).

And, the graphic information may include at least one of graphic color information (for example, a main color when configured in multi-color), graphic location information, and graphic size information.

An example of region division of the data processing unit 330 will be described with reference to FIG. 7 .

7 is an exemplary diagram in which a reference image and an image to be judged are divided into predetermined regions according to an embodiment of the present invention. As shown in FIG. 7( a ), the data processing unit 330 may divide each of the obtained image to be determined and the reference image into a first line and a second line based on letter spacing information.

In addition, as shown in FIG. 7(b) , the data processing unit 330 may divide each of the acquired image to be determined and the reference image according to the same color and spacing based on text color information and text spacing information.

Also, as shown in FIG. 7(c) , the data processing unit 330 may divide each of the obtained image to be determined and the reference image into syllable units based on the character structure information.

Referring again to FIG. 6 , the data processing unit 330 may calculate a histogram for each predetermined region of the image to be judged (S250) and calculate a histogram for each predetermined region of the reference image (S260).

Then, the data processing unit 330 may compare the histogram for each predetermined region (S270). Here, the data processing unit 330 may preferentially compare a histogram of a color channel corresponding to color information within a predetermined region among histograms of a plurality of color channels, or may compare only histograms of color channels corresponding to color information within a predetermined region. Here, color information within a predetermined area may mean a main color of text or graphics located within the predetermined area.

The histogram comparison for each region of the data processing unit 330 will be described with reference to FIG. 8 .

8 is an exemplary diagram illustrating a reference image and an image to be judged and histograms thereof according to a second embodiment of the present invention. Referring to FIG. 8, FIG. 8(a) is an image showing a divided predetermined area of a reference image in a situation in which an electric sign is operating normally, and FIG. It is an image representing a certain area.

In this case, the data processing unit 330 may calculate a histogram for each of the predetermined region of the reference image and the predetermined region of the image to be judged.

For example, the data processing unit 330 may calculate a histogram of the RGB channels of a predetermined area and compare the calculated histogram to calculate a degree of similarity, as shown in FIGS. 8(b) and 8(e).

As another example, the data processing unit 330 compares only the histogram of the red channel corresponding to the color information (red color in this example) to calculate the similarity, as shown in FIGS. 8(c) and 8(f). .

At this time, the range of error 42 between red channel histograms as shown in FIGS. 8(c) and 8(f) is greater than the error 41 between RGB channel histograms as shown in FIGS. Since the histogram is larger, according to an embodiment of the present invention, by preferentially comparing the histogram of the color channel corresponding to the color information in the predetermined area or by comparing only the histogram of the color channel corresponding to the color information in the predetermined area, the histogram can be obtained more accurately. similarity can be calculated.

Here, since the method of determining the similarity of the histogram and the method of determining whether the electric sign is malfunctioning according to the similarity determination have been described with reference to FIG. 3, a detailed description thereof will be omitted.

As described above, the method for determining whether an electric sign is malfunctioning according to the second embodiment of the present invention divides each of a reference image and an image to be judged into predetermined regions, and compares histograms calculated for each of the divided predetermined regions to determine whether the electric sign 11 ) is a method to determine whether or not there is a malfunction. The method for determining whether an electric sign is malfunctioning according to the second embodiment can be usefully used to determine malfunction of a small area in an electric sign.

According to the method for determining whether the electric sign is malfunctioning according to the second embodiment of the present invention, it is possible to determine the approximate location of the malfunctioning region in the electric sign 11 .

[Method for determining malfunction of electric signage according to the third embodiment]

9 is a flowchart illustrating a method of determining whether an electric sign malfunctions in a data processing unit according to a third embodiment of the present invention. Referring to FIG. 9 , the method for determining whether an electric sign is malfunctioning according to the third embodiment of the present invention further includes steps S330 and S340, and is different from the method for determining whether an electric sign is malfunctioning according to the first embodiment of the present invention. there is

Specifically, the data processing unit 330 divides each of the acquired image to be determined and the reference image into a predetermined number of sections for each direction based on the origin, and among the color temperature, contrast ratio, and average color value for each divided section, At least one can be produced.

Here, the histogram according to the third embodiment of the present invention, unlike the histograms of the first and second embodiments described above, the X-axis of the histogram according to the third embodiment indicates the position of a section divided by a predetermined number for each direction, and the third The Y-axis of the histogram according to the embodiment may be a modified histogram representing color temperature, contrast ratio, or average color value for each divided section.

This will be described with reference to FIGS. 10 and 11 .

Referring to FIG. 10, the data processing unit 330, as shown in FIG. 10 (b), converts the reference image shown in FIG. After creating the virtual line 54, the reference image may be divided into a predetermined number of sections 52 in the first direction based on the virtual line 54, and as in this example, 255 sections. can be divided

In addition, the data processing unit 330, as shown in FIG. 10 (c), converts the reference image shown in FIG. 10 (a) into a virtual line parallel to the first direction and perpendicular to the second direction based on the origin 51 ( 55), the reference image may be divided into a predetermined number of sections 53 in the second direction based on the imaginary line 55, and may be divided into 255 sections as in this example. .

According to this division, the base image may be divided into 255*255 sections.

In addition, the data processing unit 330 may divide the image to be determined as shown in FIG. 10(d) in the first direction and the second direction in the above-described manner.

In addition, the data processing unit 330 may calculate the color temperature of the divided sections in the first direction 52 and sections in the second direction 53 for each of the reference image and the image to be judged.

Next, as shown in FIG. 11 , the data processing unit 330 may calculate a histogram in which the X-axis indicates positions of sections divided into a predetermined number for each first or second direction, and the Y-axis indicates color temperatures for each divided section. In addition, in this example, only the histogram representing the color temperature is calculated, but the data processing unit 330 may also calculate the histogram representing the contrast ratio or average color value, if necessary.

11(a) and 11(b) are histograms showing color temperatures for each section in the first direction of the reference image and the image to be determined, and FIGS. It is a histogram showing the color temperature for each section in the second direction.

Meanwhile, the data processor 330 may compare histograms representing color temperatures for each section of the reference image and the image to be judged to calculate respective degrees of similarity. At this time, the data processing unit 330 may compare the histograms with each other to calculate the degree of similarity, or extract points and compare patterns of the points to calculate the degree of similarity.

Here, since the method of determining the similarity of the histogram and the method of determining whether the electric sign is malfunctioning according to the similarity determination have been described with reference to FIG. 3, a detailed description thereof will be omitted.

Meanwhile, the data processing unit 330 may compare the histogram of the reference image and the image to be judged to determine the location of the malfunctioning area of the electric sign (S390).

In this regard, referring to FIG. 11 , the data processing unit 330 calculates the coordinate values of the region 61 where the difference between the histogram representing the color temperature of the first direction section of the reference image and the image to be determined is [80 to 125]. can do.

In addition, the data processing unit 330 may calculate the coordinate values of the region 62 where the difference between the histogram representing the color temperature of the second direction section of the reference image and the image to be determined is [110 to 150].

Further, when the coordinate values of the section where the difference between the histograms in the first direction occurs are [80 to 125] and the coordinate values of the section where the difference between the histograms in the second direction occurs are [110 to 150], the data processing unit 330 It can be determined that a malfunction has occurred in a region having coordinates in the range of [80 to 125] and Y-axis coordinates in the range of [110 to 150] based on the origin in the electric sign 11.

As described above, the method for determining whether an electric sign is malfunctioning according to the third embodiment of the present invention indicates at least one of color temperature, contrast ratio, and average color value for each divided section of the reference image and the image to be determined. A modified histogram is calculated, and the modified histogram is compared to determine a malfunctioning area within the electric sign 11 based on the coordinates of the area where the difference between the histograms occurs.

On the other hand, the data processing unit 330 according to an embodiment of the present invention determines whether the above-described first malfunction is determined according to the state of the reference image or the image to be judged, the size of the electric sign 11, real-time resources of the image analysis system, environment information, and the like. Malfunction of the electric sign 11 may be determined by selecting at least one of the determination method, the second malfunction determination method, and the third malfunction determination method.

The data processing unit according to an embodiment of the present invention may be implemented using software, hardware, or a combination thereof. For example, according to hardware implementation, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors ), controllers, micro-controllers, micro-processors, and other electrical units for performing functions.

Also, terms such as "first", "second", "third" and "fourth" in the specification and claims, if any, are used to distinguish between similar elements, but not necessarily Used to describe a specific sequence or order of occurrence. It will be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein may be operated in sequences other than those shown or described herein, for example. Likewise, where a method is described herein as comprising a series of steps, the order of those steps presented herein is not necessarily the order in which those steps may be performed, and any recited steps may be omitted and/or here Any other step not described may be added to the method. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

In addition, terms such as "left", "right", "front", "rear", "top", "bottom", "above", "below" in the specification and claims are used for explanation, It is not necessarily intended to describe an invariant relative position. It will be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein may, for example, operate in directions other than those shown or described herein. As used herein, the term "connected" is defined as being directly or indirectly connected in an electrical or non-electrical manner. Objects described herein as "adjacent" to each other may be in physical contact with each other, in close proximity to each other, or in the same general scope or area as is appropriate for the context in which the phrase is used. The presence of the phrase “in one embodiment” herein refers to the same embodiment, although not necessarily.

In addition, in the specification and claims, 'connected', 'connected', 'engaged', 'fastened', 'coupled', 'coupled', etc., refer to various variations of these expressions directly with other components. It is used in the meaning of being connected or indirectly connected through other components.

On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

In addition, the suffixes "module" and "unit" for components used in this specification are given or used interchangeably in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves.

Also, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. Singular expressions used herein include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or some of the steps It should be construed that it may not be included, or may further include additional components or steps.

So far, the present invention has been looked at with respect to its preferred embodiments. All embodiments and conditional examples disclosed throughout this specification are described with the intention of helping readers to understand the principles and concepts of the present invention by those skilled in the art having ordinary knowledge in the technical field of the present invention, and those skilled in the art It will be understood that it can be implemented in a modified form within the range that does not deviate from the essential characteristics of the present invention.

Therefore, the disclosed embodiments should be considered from an illustrative rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope will be construed as being included in the present invention.

Meanwhile, the electronic sign error determination method according to various embodiments of the present invention described above may be implemented as a program and provided to servers or devices. Accordingly, each device may access a server or device in which the program is stored and download the program.

In addition, the method according to various embodiments of the present invention described above may be implemented as a program and stored in various non-transitory computer readable media to be provided. A non-transitory readable medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and can be read by a device. Specifically, the various applications or programs described above may be stored and provided in non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: shooting device 200: operating device
310: photographing unit 320: sensor unit

Claims (21)

A data processing unit that compares the histogram of the reference image and the histogram of the image to be judged to determine whether the electric sign is malfunctioning; image analysis system comprising a. According to claim 1,
The histogram is
The image analysis system characterized in that the calculation is based on the number of pixels corresponding to pixel values for each color channel for each of the reference image and the image to be judged. According to claim 1,
The data processing unit,
The image analysis system, characterized in that for calculating a similarity by extracting a plurality of points from each of the histogram of the reference image and the histogram of the image to be judged, and comparing a pattern of a line connecting the extracted points. According to claim 1,
The histogram is
The image analysis system according to claim 1 , wherein each of the reference image and the image to be judged is divided into predetermined regions and calculated based on the number of pixels corresponding to pixel values for each color channel for each of the divided regions. According to claim 4,
The data processing unit,
The video analysis system according to the character information and the graphic information of the traffic information included in each of the judgment subject image and the reference image, to determine the division criterion of the predetermined region. According to claim 4,
The data processing unit,
The image analysis system, characterized in that for calculating the degree of similarity by comparing the histogram calculated for each predetermined region. According to claim 6,
The data processing unit,
Among the histograms of a plurality of color channels, the similarity is calculated by preferentially comparing histograms of color channels corresponding to color information within the predetermined area or by comparing only histograms of color channels corresponding to color information within the predetermined area. video analysis system. According to claim 1,
The data processing unit,
Dividing each of the reference image and the image to be judged into a predetermined number of sections for each direction based on the origin, and calculating at least one of a color temperature, a contrast ratio, and an average color value for each of the divided sections to calculate a histogram. video analysis system. According to claim 8,
The data processing unit,
The image analysis system, characterized in that for calculating the degree of similarity by comparing the histogram for each section of the reference image and the image to be determined. According to claim 9,
The data processing unit,
The video analysis system, characterized in that for determining the location of the malfunctioning area of the electric sign by comparing the histogram of the reference image and the image to be judged. The method of claim 3, 6 or 9,
The data processing unit,
If the calculated similarity is smaller than the threshold value, the video analysis system, characterized in that it is determined that the electric sign is malfunctioning. According to claim 1,
The data processing unit,
An image analysis system characterized in that for detecting a reference image corresponding to the environmental information of the image to be judged from among a plurality of reference images. According to claim 1,
The image analysis system further comprising a; photographing unit for capturing the image to be judged. According to claim 1,
Further comprising a sensor unit for obtaining environmental information around the electric sign;
The environmental information includes at least one of fine dust, fog, heavy rain, heavy snow, humidity, temperature, season information, and time information. According to claim 1,
The data processing unit,
An image analysis system, characterized in that for generating analysis result data indicating whether the electric sign is malfunctioning. According to claim 15,
The analysis result data,
Video analysis system characterized in that it further comprises at least one of traffic information, crime prevention information and event information. According to claim 15,
An operating device providing a control function based on the analysis result data; characterized in that it further comprises a video analysis system. 18. The method of claim 17,
The video analysis system, characterized in that the operating device provides a notification enabling to check whether the electric sign is malfunctioning. An image analysis method comprising: comparing a histogram of a reference image with a histogram of an image to be judged to determine whether an electric sign is malfunctioning. A computer readable recording medium storing a program for performing the image analysis method according to claim 19. A computer program comprising program codes stored in a computer readable recording medium to execute the image analysis method according to claim 19.

Images