JP2023170051A - Marker detection device, monitoring system, and methods therefor - Google Patents

Abstract

To detect markers at high speed.SOLUTION: A marker detection device comprises: a storage unit configured to store, as a registered feature quantity, a feature quantity generated from an image obtained by imaging a marker in which a predetermined color is varied in brightness in one direction; an image acquisition unit configured to acquire a one-dimensional image; a feature quantity calculation unit configured to generate an observation feature quantity by calculating a feature quantity from the one-dimensional image; and a marker determination unit configured to determine whether or not the one-dimensional image includes the marker, on the basis of a similarity between the registered feature quantity and the observation feature quantity.SELECTED DRAWING: Figure 14

Description

The present disclosure relates to marker detection devices, monitoring systems, and methods thereof.

There is a monitoring system for inspecting structures such as tunnels. In inspections using a monitoring system, an inspection vehicle equipped with a monitoring system travels inside the structure and measures deformation of the structure in a predetermined monitoring area, thereby determining whether an abnormality has occurred in the structure. Check whether

For example, Patent Document 1 discloses a contact line metal fitting inspection system for obtaining an image of a specific contact line metal fitting attached to a contact line from a line sensor camera installed on the roof of a railway vehicle.

Patent No. 5423567

In order to efficiently inspect structures, it is desirable to run an inspection vehicle during operation. In that case, it is necessary to drive the inspection vehicle at high speed so as not to interfere with the operation. In order to recognize the monitoring area from an inspection vehicle traveling at high speed, it is necessary to detect the marker indicating the monitoring area at high speed.

In view of the above technical problems, one aspect of the present invention aims to detect markers at high speed.

In order to solve the above problems, a marker detection device according to one aspect of the present invention is configured to store a feature amount generated from an image of a marker whose predetermined color changes in brightness in one direction as a registered feature amount. an image acquisition unit configured to acquire a one-dimensional image; and a feature calculation unit configured to generate an observed feature by calculating a feature from the one-dimensional image. and a marker determination unit configured to determine whether a marker is included in the one-dimensional image based on the degree of similarity between the registered feature amount and the observed feature amount.

According to one aspect of the present invention, markers can be detected at high speed.

1 is a conceptual diagram showing an example of a monitoring system. FIG. 2 is a conceptual diagram showing an example of measuring displacement inside a tunnel. FIG. 1 is a conceptual diagram showing an example of a line scan camera. FIG. 2 is a conceptual diagram showing an example of LBP feature amounts. FIG. 2 is a conceptual diagram illustrating an example of a histogram of LBP features. FIG. 2 is a conceptual diagram illustrating an example of a 1DLBP feature amount. FIG. 2 is a conceptual diagram showing an example of a marker. FIG. 2 is a conceptual diagram showing an example of a histogram of 1DLBP feature amounts. It is a conceptual diagram which shows an example of the LBP value by one gradation. FIG. 2 is a conceptual diagram showing an example of an LBP value based on multiple gradations. FIG. 2 is a conceptual diagram showing an example of a method for optimizing the viewing angle of a camera. FIG. 1 is a diagram showing an example of the overall configuration of a monitoring system. 1 is a diagram showing an example of a hardware configuration of a computer. 1 is a diagram showing an example of a functional configuration of a monitoring system. FIG. 2 is a diagram illustrating an example of a monitoring method.

Embodiments of the present invention will be described below with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals and redundant explanation will be omitted.

[overview]
In recent years, with the development of autonomous driving technology and artificial intelligence (AI) technology, the importance of image processing technology has been increasing. Among these, marker detection technology is conventionally known as a method of checking whether a marker determined as a detection target exists in a designated image. Marker detection technology is used for applications such as triggering to turn on/off some signals, self-location estimation, or image matching. In particular, high-speed image processing is essential for marker detection on objects that move at high speed.

An example of an application of marker detection technology is a structure inspection technology. For example, in structures such as railway tunnels, it is important to carry out preventive maintenance to check for abnormalities on a daily basis. Tunnels can be deformed due to loads from above, ground pressure from below, etc., so deformation of tunnels is one of the important inspection items. If the tunnel is deforming rapidly, immediate measures must be taken to maintain the tunnel's functionality.

Traditionally, tunnel inspections were carried out manually by people entering the tracks outside of train operating hours. In recent years, various monitoring systems have been developed from the viewpoint of efficiency. Ideally, it would be possible to install it on a railway vehicle in regular operation, and to be able to monitor tunnel deformation automatically on a daily basis.

FIG. 1 is a conceptual diagram showing an example of a monitoring system. As shown in FIG. 1, the monitoring system 1 is mounted on the outside of an inspection vehicle 910, such as on the ceiling. The inspection vehicle 910 measures internal displacement of the tunnel 900 while traveling in a monitoring area 901 set in the tunnel 900 . In the monitoring area 901, markers 904-1 and 904-2 are installed at a starting point 902 and an ending point 903, respectively, so that the monitoring system 1 can recognize the monitoring area 901.

FIG. 2 is a conceptual diagram showing an example of measuring the displacement inside a tunnel. As shown in FIG. 2, in the internal displacement measurement, displacement data at five measurement points 905-1 to 905-5 on the cross section of the tunnel 900 is measured, and the displacement data is measured on four survey lines connecting each measurement point 905. The deformation of the tunnel 900 is evaluated based on the following.

A monitoring system that monitors structures while traveling as described above is called a "traveling-type monitoring system." In addition to measurement accuracy, a traveling monitoring system requires accuracy for the following measurement targets. 1. Measure only in monitoring areas that require inspection (i.e., do not collect unnecessary data). 2. Measure the same monitoring area each time (i.e., repeatedly monitor the correct target).

Regarding 1 above, there is a problem in that a monitoring system that continuously collects data for a certain period of time takes time to process the huge amount of data that has been obtained. For example, data processing such as aligning the monitoring area and extracting necessary data portions is required every time a measurement is made.

Regarding the above 2, there is a problem that it is difficult for the monitoring system itself to specify the position and start measurement in the same monitoring area. For example, methods that use radio waves such as GPS (Global Positioning System), WiFi (registered trademark), or RFID (Radio Frequency Identification) have a positioning accuracy of about several meters, so measurements are started in the same monitoring area each time. and difficult to stop.

In order to solve the above problem, an embodiment of the present invention provides a marker detection device using one-dimensional image data and a monitoring system using the marker detection device. In one embodiment, marker determination is performed using a color marker and a color line scan camera, as an example.

FIG. 3 is a conceptual diagram showing an example of a line scan camera. As shown in FIG. 3, the line scan camera 920 is a digital camera whose imaging area is a linear imaging line 921 along one predetermined direction.

As shown in FIG. 1, in the monitoring system 1 in one embodiment, markers 904-1 and 904-2 are installed on the tunnel wall surface at the starting point 902 and the ending point 903 of the monitoring area 901, and Shoot continuously. The monitoring system 1 reads one-dimensional color image data from a camera and determines whether the one-dimensional image contains a correct marker stored in advance.

If the image data includes the marker installed at the starting point 902, the monitoring system 1 starts measuring the internal displacement of the tunnel 900. On the other hand, if the image data includes the marker installed at the end point 903, the monitoring system 1 stops measuring the internal displacement of the tunnel 900.

≪LBP (Local Binary Pattern) features≫
The monitoring system in this embodiment detects markers included in a one-dimensional image using an image feature called LBP (Local Binary Pattern). LBP is a feature amount expressing local features of an image. LBP has the advantage of being resistant to fluctuations in illumination and having low calculation cost because it can be calculated using simple calculations. Details regarding LPB are disclosed in Reference 1 below.

[Reference 1] Ojala T, Pietikainen M and Harwood D, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions." Proceedings of the 12th International Conference on Pattern Recognition (ICPR 1994), Jerusalem, Israel. Vol I , pp. 582-585, 1994.

FIG. 4 is a conceptual diagram showing an example of a conventional LBP feature amount. As shown in FIG. 4, the conventional LBP feature amount is calculated by the following procedure.

First, a 3x3 pixel area (hereinafter referred to as "3x3 kernel") on the image is selected. Next, in the selected 3×3 kernel, eight vertically, horizontally and diagonally adjacent pixel values are binarized using the center pixel value as a threshold. At this time, a pixel whose value is larger or equal to that of the center pixel is set to 1, and a smaller pixel is set to 0.

In the example shown in FIG. 4, using the center pixel value 6 as the threshold, (7, 13, 28) in the first row becomes (1, 1, 1), and (6, 6, 11) in the second row. ) becomes (1, -, 1), and (5, 59, 1) in the third row becomes (0, 1, 0).

Next, the eight binarized pixels are regarded as an 8-bit code string starting from the upper left pixel and extending clockwise to obtain a binary LBP value. In the example shown in FIG. 4, an LBP value of (1, 0, 1, 0, 1, 1, 1, 1) is obtained.

Subsequently, the binary LBP value is converted into a decimal number. Then, the obtained decimal LBP value is used as the LBP feature amount corresponding to the center pixel. In the example shown in FIG. 4, 175 is obtained by converting (1, 0, 1, 0, 1, 1, 1, 1) into a decimal number.

In this way, the 3×3 kernel is moved and the LBP value is calculated for the entire image. As a result, a histogram is obtained in which the LBP values obtained over the entire image are distributed. By comparing the histograms of LBP values obtained from two images, the degree of similarity between the images can be determined.

FIG. 5 is a diagram showing an example of a histogram of LBP values. FIG. 5(A) is an example of an image used to calculate the LBP value, and FIG. 5(B) is a histogram of the LBP value calculated from the image shown in FIG. 5(A).

The monitoring system in this embodiment uses one-dimensional image data captured by a line scan camera or the like. Therefore, a 1DLBP (One Dimensional Local Binary Pattern) feature amount, which is a one-dimensional LBP, is obtained. Details regarding 1DLBP are disclosed in Reference 2 below.

[Reference 2] Lotfi Houam, Adel Hafiane, Abdelhani Boukrouche, Eric Lespessailles, and Rachid Jennane, "One Dimensional Local Binary Pattern for Bone Texture Characterization", Pattern Analysis and Applications, vol. 17(1), pp. 179-193 , 2014.

FIG. 6 is a conceptual diagram showing an example of 1DLBP feature amounts in this embodiment. As shown in FIG. 6, the 1DLBP feature amount in this embodiment is calculated using the following procedure.

First, select a 9 pixel area on the image. Next, among the nine selected pixels, each of the four consecutive pixels above and below is binarized using the center pixel value as a threshold. At this time, a pixel whose value is larger or equal to that of the center pixel is set to 1, and a smaller pixel is set to 0.

In the example shown in Figure 6, using the center pixel value 6 as the threshold, the four consecutive pixels above (7, 13, 28, 11) become (1, 1, 1, 1), and the four consecutive pixels below become (1, 1, 1, 1). The four pixels (1,59,5,6) become (0,1,0,1).

Next, the eight binarized pixels are regarded as an 8-bit code string to obtain a binary LBP value. In the example shown in FIG. 6, an LBP value of (1, 0, 1, 0, 1, 1, 1, 1) is obtained.

Subsequently, the binary LBP value is converted into a decimal number. Then, the obtained decimal LBP value is used as the LBP feature amount corresponding to the center pixel. In the example shown in FIG. 6, 175 is obtained by converting (1, 0, 1, 0, 1, 1, 1, 1) into a decimal number.

In this way, the 9-pixel area to be selected is moved and the LBP value is calculated for the entire image. As a result, a histogram is obtained in which the LBP values obtained over the entire image are distributed.

≪Marker≫
FIG. 7 is a conceptual diagram showing an example of a marker in this embodiment. As shown in FIG. 7, marker 904 includes a linear gradation in which a predetermined color varies in intensity in one direction.

Marker 904 includes one or more linear gradations. The number of linear gradations included in marker 904 and the length of each linear gradation are optimized based on the resolution of the line scan camera. The optimization method will be described later.

For example, the marker in this embodiment is configured such that in an RGB color model image, a gradation in which the brightness of R is constant and the brightness of G and B increases linearly from 0 to 255 occurs eight times in a row.

FIG. 8 is a conceptual diagram showing an example of a histogram generated from markers in this embodiment. In FIG. 8, LBP values are calculated for each RGB from the markers illustrated in FIG. 7 and normalized to create a histogram.

As mentioned above, since the brightness of R is constant at 0, all values become 1 when binarized, and the LBP value becomes 255. Therefore, the histogram regarding R has a shape in which most of the bins are concentrated in 255 bins. Also, since the brightness of G and B increases linearly from 0 to 255, when binarized, the 5th to 8th bits corresponding to the lower 4 pixels from the center are 1, except for the part where the brightness switches from 255 to 0. Therefore, the LBP value is 240. Therefore, the histogram for G and B has a shape in which most of the histograms are concentrated in 240 bins.

As described above, the monitoring system in this embodiment employs a design in which a bias appears in the histogram of LBP values. This enables differentiation from histograms generated from images other than markers, and improves the accuracy of marker detection.

The marker in this embodiment may be a grayscale image. In this case, since only one histogram is generated, the amount of calculation can be reduced, but the accuracy may be lower than when a histogram is generated for each RGB.

Here, the reason why the marker design is such that the gradation is continuous multiple times will be explained with reference to FIGS. 9 and 10.

FIG. 9 is a conceptual diagram showing an example of LBP values based on one gradation. As shown in Figure 9, in an image of a marker consisting of one gradation taken with a 1024-pixel line scan camera, the same brightness value will occur approximately 4 times in a row because 1024÷256=4. Become. At this time, for example, if 9 pixels of (0, 0, 0, 1, 1, 1, 1, 2, 2) are binarized, the LBP value will be 248.

Originally, for colors whose brightness changes, the histogram is intended to have a shape that is concentrated in 240 bins. However, if the same brightness values continue as described above, the shape will be dispersed into a plurality of bins. Such a phenomenon occurs more frequently as the number of pixels acquired increases when a marker is photographed at the full viewing angle of the camera.

FIG. 10 is a conceptual diagram illustrating an example of LBP values based on multiple gradations. As shown in Figure 10, in an image taken with a 1024 pixel line scan camera of a marker with 8 consecutive gradations, the same brightness value is continuous because 1024 ÷ 8 ÷ 256 = 0.5. Never. At this time, the LBP value is 240 in areas other than the area where the brightness value changes, such as (..., 254, 255, 0, 1, 2, . . .).

As explained above, in the marker in this embodiment, the gradation is repeated multiple times so that the same brightness value does not continue. Specifically, the gradation is continued a number of times equal to or greater than the value obtained by dividing the number of pixels of the line scan camera by 256 (luminance value).

Note that the number of gradations included in the marker and the length of each gradation need to be optimized depending on the field of view (FOV) of the camera. The monitoring system in this embodiment calculates the LBP value for the entire image and creates a histogram. Therefore, if the image includes a portion other than the marker, the histogram of LBP values changes, and marker detection accuracy decreases.

Therefore, it is necessary to adjust the viewing angle of the line scan camera so that the marker is photographed over the entire viewing angle of the line scan camera. Furthermore, it is necessary to set the number of gradations so that the brightness values are not continuous at that viewing angle.

FIG. 11 is a conceptual diagram showing an example of a method for optimizing the viewing angle of a camera. FIG. 11(A) is an example in which a marker is photographed in a part of the viewing angle of the line scan camera. As shown in FIG. 11A, when a marker is photographed in a part of the viewing angle, a background 906 in addition to the marker 904 is reflected in the image. As a result, the histogram of LBP values generated from the acquired image is different from the histogram generated from the image of markers only.

FIG. 11(B) is an example in which markers are photographed over the entire viewing angle of the line scan camera. As shown in FIG. 11B, when markers are photographed over the entire viewing angle, only the marker 904 will be visible in the image. As a result, the histogram of LBP values generated from the acquired image will be the same or similar to the histogram generated from the marker-only image.

By using a telecentric lens in a line scan camera, the above optimization can be easily performed. A telecentric lens is a lens that can project a subject at a constant size regardless of the distance to the subject.

The monitoring system 1 in one embodiment allows an algorithm for detecting markers to be implemented in hardware capable of parallel processing, such as an FPGA (Field Programmable Gate Array). Thereby, the monitoring system 1 in one embodiment can perform image processing at high speed in real time.

[Embodiment]
One embodiment of the present invention is a monitoring system that monitors structures such as tunnels. The monitoring system is installed on a mobile object such as an inspection vehicle that inspects a structure while driving. The monitoring system regularly and continuously photographs a predetermined position of a structure, and controls the measurement of the structure to start or stop when a marker is detected from the photographed image.

<Overall configuration of monitoring system>
First, the overall configuration of the monitoring system in this embodiment will be described with reference to FIG. 12. FIG. 12 is a block diagram showing an example of the overall configuration of the monitoring system in this embodiment.

As shown in FIG. 12, the monitoring system 1 in this embodiment includes an imaging device 10, a marker detection device 20, and a measurement device 30. The photographing device 10 and the marker detection device 20, and the marker detection device 20 and the measurement device 30 are each electrically connected.

In the monitoring system 1, the photographing device 10, the marker detecting device 20, and the measuring device 30 may each be realized as separate devices, or the monitoring system 1 may be realized as a single device that combines the functions that the photographing device 10, the marker detecting device 20, and the measuring device 30 should have. It may also be realized as a single monitoring device.

The photographing device 10 is an electronic device that acquires one-dimensional image data (hereinafter also referred to as "one-dimensional image") photographing a predetermined position of a structure. An example of the imaging device 10 is a color line scan camera.

The marker detection device 20 is an information processing device such as a PC (Personal Computer), a workstation, or a server that detects a predetermined marker from a one-dimensional image acquired by the imaging device 10. The marker detection device 20 acquires a one-dimensional image from the imaging device 10 and detects a predetermined marker from the one-dimensional image. The marker detection device 20 transmits a control signal instructing the measurement device 30 to start or stop measurement based on the marker detection results.

The measuring device 30 is a device that includes a measuring section such as a laser distance sensor that measures the state of a structure, and a storage section such as a PC (Personal Computer), workstation, or server that stores measurement results. The measurement device 30 receives a control signal from the marker detection device 20 and starts or stops measuring the structure according to the control signal.

Note that the overall configuration of the monitoring system 1 shown in FIG. 1 is an example, and there may be various system configuration examples depending on the usage and purpose. For example, the marker detection device 20 may be implemented in a programmable processing circuit such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit), and may be incorporated into the imaging device 10 or the measurement device 30.

<Hardware configuration of monitoring system>
Next, the hardware configuration of the monitoring system 1 in this embodiment will be explained with reference to FIG. 13.

≪Computer hardware configuration≫
The marker detection device 20 and measurement device 30 in this embodiment are realized by, for example, a computer. FIG. 13 is a block diagram showing an example of the hardware configuration of the computer 500 in this embodiment.

As shown in FIG. 13, the computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an HDD (Hard Disk Drive) 504, an input device 505, It has a display device 506, a communication I/F (Interface) 507, and an external I/F 508. CPU501, ROM502, and RAM503 form what is called a computer. Each piece of hardware in the computer 500 is interconnected via a bus line 509. Note that the input device 505 and the display device 506 may be connected to an external I/F 508 for use.

The CPU 501 is an arithmetic unit that implements control and functions of the entire computer 500 by reading programs and data from a storage device such as the ROM 502 or the HDD 504 onto the RAM 503 and executing processing.

The ROM 502 is an example of a nonvolatile semiconductor memory (storage device) that can retain programs and data even when the power is turned off. The ROM 502 functions as a main storage device that stores various programs, data, etc. necessary for the CPU 501 to execute various programs installed on the HDD 504 . Specifically, the ROM 502 stores data such as boot programs such as BIOS (Basic Input/Output System) and EFI (Extensible Firmware Interface) that are executed when the computer 500 is started, OS (Operating System) settings, and network settings. is stored.

The RAM 503 is an example of a volatile semiconductor memory (storage device) whose programs and data are erased when the power is turned off. The RAM 503 is, for example, DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). The RAM 503 provides a work area where various programs installed on the HDD 504 are expanded when the CPU 501 executes them.

The HDD 504 is an example of a nonvolatile storage device that stores programs and data. The programs and data stored in the HDD 504 include an OS, which is basic software that controls the entire computer 500, and applications that provide various functions on the OS. Note that, instead of the HDD 504, the computer 500 may use a storage device (for example, SSD: Solid State Drive) that uses flash memory as a storage medium.

The input device 505 is a touch panel used by the user to input various signals, operation keys or buttons, a keyboard or mouse, a microphone used to input sound data such as voice, or the like.

The display device 506 includes a display such as a liquid crystal or organic EL (Electro-Luminescence) that displays a screen, a speaker that outputs sound data such as audio, and the like.

The communication I/F 507 is an interface that connects to a communication network and allows the computer 500 to perform data communication.

External I/F 508 is an interface with an external device. The external device includes a drive device 510 and the like.

The drive device 510 is a device for setting the recording medium 511. The recording medium 511 here includes a medium that records information optically, electrically, or magnetically, such as a CD-ROM, a flexible disk, and a magneto-optical disk. Further, the recording medium 511 may include a semiconductor memory or the like that electrically records information, such as a ROM or a flash memory. Thereby, the computer 500 can read and/or write to the recording medium 511 via the external I/F 508.

Note that the various programs installed on the HDD 504 are, for example, when the distributed recording medium 511 is set in the drive device 510 connected to the external I/F 508, and the various programs recorded on the recording medium 511 are read by the drive device 510. It is installed by Alternatively, various programs to be installed on the HDD 504 may be installed by being downloaded from a network different from the communication network via the communication I/F 507.

<Functional configuration of monitoring system>
Next, the functional configuration of the monitoring system in this embodiment will be explained with reference to FIG. 14. FIG. 14 is a block diagram showing an example of the functional configuration of the monitoring system 1 in this embodiment.

≪Photography equipment≫
As shown in FIG. 14, the photographing device 10 in this embodiment includes a photographing section 11.

The photographing unit 11 photographs a predetermined position of the structure and generates a one-dimensional image. The imaging unit 11 transmits the generated one-dimensional image to the marker detection device 20.

≪Marker detection device≫
As shown in FIG. 14, the marker detection device 20 in this embodiment includes an image acquisition section 21, a feature amount calculation section 22, a similarity calculation section 23, a marker determination section 24, a measurement control section 25, and a feature amount storage. 200.

The image acquisition unit 21, the feature amount calculation unit 22, the similarity calculation unit 23, the marker determination unit 24, and the measurement control unit 25 perform processing that is executed by the CPU 501 by the program loaded from the HDD 504 onto the RAM 503 shown in FIG. realized by

The feature amount storage unit 200 stores feature amounts generated from predetermined markers. The feature amount in this embodiment is a 1DLBP feature amount. Hereinafter, the 1DLBP feature stored in the feature storage unit 200 will also be referred to as a "registered feature." The feature amount storage unit 200 is realized by the RAM 503 or the HDD 504 shown in FIG.

The image acquisition unit 21 acquires a one-dimensional image from the imaging device 10. The image acquisition unit 21 may acquire a one-dimensional image by receiving a one-dimensional image transmitted by the imaging device 10, or may obtain a one-dimensional image by requesting a one-dimensional image from the imaging device 10. may be obtained.

The feature calculation unit 22 calculates a 1DLBP feature from the one-dimensional image acquired by the image acquisition unit 21. If the one-dimensional image is a color image, the feature calculation unit 22 may calculate a 1DLBP feature for each primary color. The 1DLBP feature amount generated from the one-dimensional image acquired by the image acquisition unit 21 is also referred to as "observed feature amount."

The feature amount calculation unit 22 may perform noise removal before calculating the 1DLBP feature amount from the one-dimensional image. If the one-dimensional image contains little noise, noise removal may be omitted.

The similarity calculation unit 23 calculates the similarity between the observed feature calculated by the feature calculation unit 22 and the registered feature stored in the feature storage 200. When the feature calculation unit 22 calculates a 1DLBP feature for each primary color, the similarity calculation unit 23 calculates the similarity for each primary color. An example of similarity in this embodiment is the Euclidean distance between histograms of each feature amount. Another example of the degree of similarity in this embodiment is the correlation between histograms of each feature amount.

The marker determination unit 24 determines whether a marker is included in the one-dimensional image based on the similarity calculated by the similarity calculation unit 23. The marker determination unit 24 determines that a marker is included in the one-dimensional image when the degree of similarity is greater than or equal to a predetermined threshold. When the similarity calculation unit 23 calculates the similarity for each primary color, the marker determination unit 24 determines that the one-dimensional image includes a marker when the similarity is equal to or greater than a threshold value for all primary colors.

The measurement control unit 25 transmits a control signal instructing the measurement device 30 to start or stop measurement based on the determination result by the marker determination unit 24. The measurement control unit 25 transmits a control signal when the determination result indicates that the one-dimensional image includes a marker. On the other hand, the measurement control unit 25 does not transmit the control signal when the determination result indicates that the one-dimensional image does not include a marker.

≪Measuring device≫
As shown in FIG. 14, the measuring device 30 in this embodiment includes a measuring section 31 and a measurement result storage section 300.

The measurement unit 31 starts or stops measuring the structure according to the control signal received from the marker detection device 20. When the measurement unit 31 receives a control signal while not performing measurement, it starts measurement. On the other hand, when the measurement unit 31 receives a control signal while performing measurement, it stops the measurement. The measurement unit 31 stores the measurement results obtained by performing the measurement in the measurement result storage unit 300.

The measuring unit 31 is realized by a measuring device such as a laser distance sensor connected to the external I/F 508 shown in FIG.

The measurement result storage unit 300 stores measurement results obtained by the measurement unit 31. The measurement result storage unit 300 is realized by the RAM 503 or HDD 504 shown in FIG.

<Monitoring system processing procedure>
Next, the processing procedure of the monitoring method executed by the monitoring system 1 in this embodiment will be explained with reference to FIG. 15. FIG. 15 is a flowchart illustrating an example of the monitoring method in this embodiment.

In step S1, the photographing unit 11 included in the photographing device 10 photographs a predetermined position of the structure and generates a one-dimensional image. The photographing unit 11 continuously repeats photographing at predetermined time intervals. The photographing interval may be determined according to the relative speed of the inspection vehicle and the structure, but it is preferable to set it as short as possible.

Next, the imaging unit 11 transmits the generated one-dimensional image to the marker detection device 20. The imaging unit 11 may transmit the one-dimensional image to the marker detection device 20 every time it generates a one-dimensional image, or may transmit the latest one-dimensional image every time it receives a one-dimensional image acquisition request from the marker detection device 20. The image may be transmitted to the marker detection device 20.

In the marker detection device 20, the image acquisition unit 21 receives a one-dimensional image from the imaging device 10. The image acquisition unit 21 sends the received one-dimensional image to the feature calculation unit 22.

In step S2, the feature calculation section 22 included in the marker detection device 20 receives a one-dimensional image from the image acquisition section 21. Next, the feature calculation unit 22 performs noise removal on the received one-dimensional image. Noise removal may be performed, for example, by smoothing using a moving average filter or the like. If the one-dimensional image contains little noise, noise removal may be omitted.

Subsequently, the feature quantity calculation unit 22 generates an observed feature quantity by calculating a 1DLBP feature quantity from the one-dimensional image. Then, the feature amount calculation unit 22 sends the generated observed feature amount to the similarity calculation unit 23.

If the one-dimensional image is a color image, the feature calculation unit 22 may calculate a 1DLBP feature for each primary color. For example, if the one-dimensional image is an image of an RGB color model, 1DLBP feature amounts are calculated for each of RGB.

In step S3, the similarity calculation unit 23 included in the marker detection device 20 receives the observed feature amount from the feature amount calculation unit 22. Next, the similarity calculation unit 23 reads the registered feature amount from the feature amount storage unit 200. Subsequently, the similarity calculation unit 23 calculates the similarity between the received observed feature amount and the read registered feature amount. Then, the similarity calculation unit 23 sends the calculated similarity to the marker determination unit 24.

The degree of similarity in this embodiment is the Euclidean distance between histograms. In this case, the similarity calculation unit 23 first generates a histogram from the received observed feature amount. Further, the similarity calculation unit 23 generates a histogram from the read registered feature amount. Then, the similarity calculation unit 23 calculates the Euclidean distance between the histogram of the observed feature amount and the histogram of the registered feature amount.

When the feature amount calculation unit 22 calculates a 1DLBP feature amount for each primary color, the similarity calculation unit 23 generates a histogram for each primary color from the observed feature amount and registered feature amount. Next, the similarity calculation unit 23 calculates the Euclidean distance between histograms for each primary color.

In step S4, the marker determination unit 24 included in the marker detection device 20 receives the similarity degree from the similarity calculation unit 23. Next, the marker determination unit 24 determines whether or not the one-dimensional image includes a marker based on the received degree of similarity. Subsequently, the marker determination section 24 sends the determination result to the measurement control section 25.

When the similarity calculation unit 23 calculates the similarity for each primary color, the marker determination unit 24 determines that the one-dimensional image includes a marker when the similarity is equal to or greater than a threshold value for all primary colors.

In step S5, the measurement control unit 25 included in the marker detection device 20 receives the determination result from the marker determination unit 24. Next, the measurement control unit 25 transmits a control signal to the measurement device 30 when the received determination result indicates that a marker has been detected. On the other hand, the measurement control unit 25 does not transmit the control signal when the determination result indicates that no marker was detected.

In the measuring device 30, the measuring section 31 receives a control signal from the marker detecting device 20. When the measurement unit 31 receives a control signal while not performing measurement, it starts measurement. On the other hand, when the measurement unit 31 receives a control signal while performing measurement, it stops the measurement.

After that, the measurement unit 31 stores the measurement results obtained by the measurement in the measurement result storage unit 300. The measurement results include measurement time, measurement position, measurement value, etc. The measurement position can be acquired based on the marker detected by the marker detection device 20. The measurement result may include identification information indicating the marker instead of the measurement value.

<Effects of embodiment>
The marker determination device in this embodiment calculates a feature amount from a one-dimensional image of a marker whose predetermined color changes in brightness in one direction, and determines whether the marker is in the one-dimensional image based on the similarity with the correct feature amount. Determine whether it is included. By detecting markers based on one-dimensional images, the amount of calculation can be significantly reduced. Therefore, according to the marker determination device in this embodiment, markers can be detected at high speed.

In particular, the marker determination device in this embodiment employs a marker design that can use a 1DLBP feature amount that is one-dimensionalized from a conventional LBP feature amount, reducing the calculation cost for determining the feature amount. Can be done.

Furthermore, the marker determination device in this embodiment calculates the feature amount for each primary color and calculates the similarity of the feature amounts for each primary color, so the accuracy of marker detection is improved.

In this way, according to the marker determination device of this embodiment, the accuracy of marker detection is improved and consistency of monitoring targets can be obtained. Therefore, by using the marker determination device of this embodiment, a highly accurate and high-speed traveling type monitoring system can be realized.

The monitoring system in this embodiment reduces the time required for position marking and measurement setup in conventional manual inspections. Therefore, according to the monitoring system in this embodiment, monitoring can be performed during operation, and inspection efficiency is significantly improved.

[supplement]
Each function of the embodiments described above can be realized by one or more processing circuits. Here, the term "processing circuit" as used herein refers to a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function explained above. This includes devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications or variations can be made within the scope of the gist of the present invention as described in the claims. Changes are possible.

1 Monitoring system 10 Photographing device 11 Photographing section 20 Marker detection device 21 Image acquisition section 22 Feature value calculation section 23 Similarity calculation section 24 Marker determination section 25 Measurement control section 200 Feature amount storage section 30 Measuring device 31 Measurement section 300 Measurement result storage Department

Claims (9)

a storage unit configured to store a feature amount generated from an image of a marker whose predetermined color changes in brightness in one direction as a registered feature amount;
an image acquisition unit configured to acquire a one-dimensional image;
a feature amount calculation unit configured to generate observed feature amounts by calculating feature amounts from the one-dimensional image;
a marker determination unit configured to determine whether or not the marker is included in the one-dimensional image based on the degree of similarity between the registered feature amount and the observed feature amount;
A marker detection device comprising: The marker detection device according to claim 1,
The marker repeats a brightness change such that the brightness of adjacent pixels in the one-dimensional image is different.
Marker detection device. The marker detection device according to claim 2,
The feature amount is a 1DLBP feature amount,
Marker detection device. The marker detection device according to claim 3,
The feature amount calculation unit is configured to calculate the feature amount for each primary color constituting the marker,
The marker determination unit is configured to determine that the one-dimensional image includes the marker when all of the similarities for each of the primary colors are equal to or higher than a predetermined threshold.
Marker detection device. The marker detection device according to claim 4,
The similarity is a Euclidean distance of a histogram based on the feature amount,
Marker detection device. A monitoring system that measures the state of a structure in which a marker is installed on a moving body and whose brightness changes in a predetermined color in a direction perpendicular to the moving direction of the moving body,
The monitoring system includes:
a photographing device that photographs a one-dimensional image including a predetermined position of the structure;
a marker detection device that detects the marker from the one-dimensional image;
a measuring device that measures the state of the structure;
including;
The marker detection device includes:
a storage unit that stores a feature quantity generated from an image of the marker as a registered feature quantity;
an image acquisition unit configured to acquire the one-dimensional image;
a feature amount calculation unit configured to generate observed feature amounts by calculating feature amounts from the one-dimensional image;
a marker determination unit configured to determine whether or not the marker is included in the one-dimensional image based on the degree of similarity between the registered feature amount and the observed feature amount;
a measurement control unit configured to transmit a control signal instructing the measurement device to start or stop measurement;
A monitoring system equipped with 7. The monitoring system according to claim 6,
The marker continues the gradation a number of times equal to or more than a value obtained by dividing the number of pixels of the photographing device by a luminance value.
monitoring system. The computer is
a storage procedure for storing a feature amount generated from an image of a marker whose predetermined color changes in brightness in one direction as a registered feature amount;
an image acquisition procedure for acquiring a one-dimensional image;
a feature amount calculation procedure for generating an observed feature amount by calculating a feature amount from the one-dimensional image;
a marker determination procedure for determining whether the marker is included in the one-dimensional image based on the degree of similarity between the registered feature amount and the observed feature amount;
A marker detection method that performs. A monitoring method performed by a monitoring system that measures the state of a structure in which a marker is installed on a moving body and whose brightness changes in a predetermined color in a direction perpendicular to the moving direction of the moving body, the method comprising:
The monitoring system includes:
a photographing device that photographs a one-dimensional image including a predetermined position of the structure;
a marker detection device that detects the marker from the one-dimensional image;
a measuring device that measures the state of the structure;
including;
The marker detection device includes:
a storage procedure for storing a feature amount generated from an image of the marker as a registered feature amount;
an image acquisition procedure for acquiring the one-dimensional image;
a feature amount calculation procedure for generating an observed feature amount by calculating a feature amount from the one-dimensional image;
a marker determination procedure for determining whether the marker is included in the one-dimensional image based on the degree of similarity between the registered feature amount and the observed feature amount;
a measurement control procedure for transmitting a control signal instructing the measurement device to start or stop measurement;
Monitoring method to perform.

Images