JP2007256223A - Structure abnormality determination system, structure abnormality determination method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a structure abnormality determination system, a structure abnormality determination method, and a program, capable of determining in detail a cause classification of abnormality in a structure, by using an observed image and a displacement gage data in combination. <P>SOLUTION: A structure abnormality determination device 2b prepares a background image data in a time, based on an input image data from a monitoring camera 4, to be stored in a background image accumulation part in an inside of the structure abnormality determination device 2b. A displacement of the monitoring camera 4 and a displacement of the structure are determined based on a difference between a background image in the time and the background image in the just near past stored in the background image accumulation part. The displacement of the monitoring camera 4 and the displacement of the structure and system abnormality are also determined based on a determination result by an image processing part inside the device and a measured value by a displacement gage 1. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

 The present invention relates to a structure abnormality determination system, a structure abnormality determination method, and a program for detecting an abnormality in a large structure such as a bridge.

 Large structures such as dams, bridges, and piers are subject to large displacements and partial movements (hereinafter referred to as displacements) to structurally weak points (hereinafter referred to as movable parts) such as member connections due to secular changes and earthquakes and other disasters. May be dangerous and difficult to continue using. In that case, prompt judgment and response are required to prevent secondary disasters and minimize the impact.

 Taking a bridge as shown in FIG. 16 as an example, the basic configuration is a structure in which a bridge girder 14 is mounted on an abutment 11 or a pier 12 via a support 13, and the bridge girder 14 is moved or dropped during an earthquake. In many cases, traffic is impossible. In the event of a wide-area earthquake, it is necessary to quickly determine whether or not the bridge can pass in order to confirm the soundness of the road network. It is important to collect and judge information.

 Various methods have been devised as methods for monitoring and detecting the displacement of the structure, and there are the following methods for detecting the displacement of the structure by comparing the differences of the observed images. There is a method of detecting a displacement of a structure by installing a target marker on a large structure and tracking its position (see, for example, Patent Document 1). In addition, there is a method of detecting a displacement of a structure by extracting a characteristic partial image from an observation image of the structure and performing pattern matching or the like (see, for example, Patent Document 2). In addition, there is a method of capturing a three-dimensional structure of a structure such as a bridge girder with a laser scanner (see, for example, Patent Document 3).

Further, as a displacement detection technique using a contact sensor, a method of measuring a displacement by installing a displacement meter on a movable part is known (for example, see Non-Patent Document 1). In this method, since only the movable part installs the measuring device, it is not necessary to install another device at a point away from the target structure.
JP 2003-240550 A JP 2001-241919 A JP 2004-325209 A NTT Technical Journal, "Development of road disaster monitoring system using optical fiber sensing technology", October 2003

 However, problems common to the techniques described in Patent Documents 1 to 3 include the following. In the technologies described in Patent Documents 1 to 3, it is assumed that the observation point is installed at a fixed point such as on the ground away from the structure, and it is usually difficult to secure such a place. When a device such as a surveillance camera is installed on a structure, the observation device itself moves due to the abnormality of the structure, which makes it difficult to detect the abnormality.

 In addition, the surveillance camera image information can be monitored over a relatively wide area within the shooting range, but it also detects changes that are different from structural abnormalities, such as changes in the exterior and the occurrence of deposits and falling objects. There was a problem that it might be.

 The technique described in Patent Document 1 has a problem that only the displacement of the place where the target marker is installed can be tracked. Further, the displacement detection by the displacement meter has a problem that only the local displacement of the installation location can be detected. In addition, when the entire structure is displaced for each installation location, there is also a problem that it cannot be detected.

 The present invention has been made to solve such a problem, the surveillance camera is installed on a structure, detects the displacement of the camera from the photographed image by comparing a stationary part such as a ground part and a close-up view of the structural part, In addition, the object is to accurately determine the presence or absence of displacement of the structure. Another object of the present invention is to realize detailed determination of the type of abnormality cause by combining information of the displacement meter with abnormality detection information of the monitoring camera.

 The present invention has been made to solve the above-described problems. The invention according to claim 1 is directed to an imaging unit for imaging a structure and its surroundings, and a displacement measuring unit installed in the structure. A structure displacement determining means for determining a displacement state of the structure from the image data photographed by the photographing means and the displacement data measured by the displacement measuring means, the structure displacement determining means, Comparing image data for each time taken by the photographing means to determine the displacement state of the structure, the judgment result of the image comparison judging means, and the displacement data of the displacement measuring means And an abnormality determining means for determining a displacement state of the structure and a system abnormality state.

 The invention according to claim 2 is the structure abnormality determination system according to claim 1, wherein the image data is averaged image data averaged for a predetermined time.

 Further, in the invention according to claim 3, the image comparison means divides the image data into a distant view region and a foreground region, and compares the image data for each time for each region. The structure abnormality determination system according to claim 1 or 2, wherein a moving state of the photographing unit is determined.

 According to a fourth aspect of the present invention, the image comparison and determination means determines the displacement state of the structure by calculating a displacement difference at a predetermined corresponding point in the image data for each time. It is a structure abnormality discrimination system in any one of Claims 1-3 characterized by the above-mentioned.

 According to a fifth aspect of the present invention, there is provided a first step in which the photographing unit photographs the structure and the surrounding situation, the displacement measuring unit measures the displacement data of the structure, and the image comparison determination unit includes the photographing. A second step of comparing the image data for each time taken by the means to determine the displacement state of the structure; the abnormality determination means; the determination result of the image comparison determination means; and the displacement data of the displacement measurement means And a third step of determining the displacement state of the structure and the system abnormality state.

 Further, the invention described in claim 6 is a photographing means for photographing the structure and its surroundings, a displacement measuring means installed in the structure, image data photographed by the photographing means, and the displacement A computer program for performing a structure displacement determination process in a structure abnormality determination system comprising a structure displacement determination means for determining a displacement state of the structure from displacement data measured by a measurement means, the imaging means From the image comparison determination step for comparing the image data taken every time and determining the displacement state of the structure, the determination result in the image comparison determination step, and the displacement data of the displacement measuring means, An abnormality determination step for determining a displacement state and a system abnormality state is a computer program for causing a computer to execute.

 According to the present invention, it is possible to accurately determine the presence or absence of displacement of the structure in a form in which the monitoring camera is installed on the structure such as a bridge.

 Since the observation image and the displacement meter data are used in combination, the reliability is higher and the cause type of the abnormality can be determined in detail.

 Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram showing the overall configuration of the structure abnormality determination system according to the present embodiment. The structure abnormality determination system in FIG. 1 includes a displacement meter 1 that measures a displacement amount between bridge girders, a monitoring center 2, an image transmission device 3, and a monitoring camera 4 that captures an image of a bridge. . The monitoring center 2 receives the displacement data from the displacement meter 1, the displacement data from the analyzer 2 a, and the image data sent from the monitoring camera 4 via the image transmission device 3. It is comprised from the structure abnormality determination apparatus 2b.

 FIG. 2 is a functional block diagram showing the processing and data flow of the structure abnormality determination device 2b in the structure abnormality determination system according to this embodiment. As shown in FIG. 2, the camera image input unit 201 receives video data and sends the data to the background image creation unit 202, and the background image creation unit 202 creates background image data from the received video data.

 The generated background image data is stored in the background image storage unit 203. Based on the stored data, the image processing unit 204 determines the bridge, the camera, and the background image data X from the past and the background image data Y at the time. The position of is determined. The determination result in the image processing unit 204 is stored in the image processing result memory unit 205. On the other hand, displacement data measured by the displacement meter 1 is received by the displacement meter measurement value input unit 206, and the abnormality determination unit 207 receives the determination result stored in the image processing result memory unit 205 and the displacement meter measurement value input unit. A detailed abnormality determination is performed in combination with the displacement data received at 206. The result display unit 208 displays the result of abnormality determination.

 FIG. 3 is a detailed functional block diagram of the background image creation unit 202. The background image creation unit 202 includes a background image calculation unit 202a, an averaged image memory unit 202b that stores an averaged image, and an averaged time memory unit 202c that stores an elapsed time of a captured image used for averaging. Has been. The background image creation unit 202 creates a new averaged image from the video captured by the monitoring camera 4 and the latest averaged image at regular time intervals, and stores them in the averaged image memory unit 202b. When the predetermined separation time is reached, the created averaged image is output to the background image storage unit 203, and the averaged image memory unit 202b and the averaged time memory unit 202c are cleared.

 FIG. 4 is a detailed functional block diagram of the image processing unit 204. The distant view / near view division processing unit 204a in the image processing unit 204 acquires the latest past background image data X and the background image data Y at the time from the background image storage unit 203, and the distant view image data Xf and Yf and the foreground respectively. The image data is divided into Xc and Yc. For example, with respect to the background image in FIG. 5, the most recent background image X is divided into the foreground image data Xc and the distant view image data Xf by mask processing, and the background image Y at that time is divided into the foreground image data Yc and the distant view image data. Divide into Yf. Here, the photographed image is a large structure with almost no displacement in normal times, and the division position may be fixed by mask processing.

 The distant view memory unit 204b stores the distant view image data, and the foreground image unit 204e stores the foreground image data. The distant view comparison processing unit 204c compares the distant view image data at the time with the distant view image data of the latest past, and calculates the displacement amount and the displacement direction. For example, for comparison of distant view image data as shown in FIG. 6, distant view image data Xf and Yf stored in the distant view memory unit 204b are compared by the distant view comparison processing unit 204c, and images B1 and B2 of the same building are compared. The displacement amount A1 and the displacement angle θ1 are calculated. In the distant view comparison result memory unit 204d, the comparison result in the distant view comparison processing unit 204c is stored.

 The foreground memory unit 204e stores the foreground image data Yc of the current time and the most recent foreground image data Xc divided by the distant view / near view division processing unit 204a. The foreground comparison processing unit 204f compares the foreground image data Yc at the time with the foreground image data Xc of the past past to obtain the displacement amount and the displacement direction. For example, for comparison of the foreground image data as shown in FIG. 7, the foreground image data Xc and Yc stored in the foreground memory unit are compared by the foreground comparison processing unit 204f, and the same center broken line images C1 and C2 are compared. The displacement amount A2 and the displacement angle θ2 are calculated. In the foreground comparison result memory unit 204g, the comparison result in the foreground comparison processing unit 204f is stored.

 The comprehensive comparison determination unit 204h integrates the comparison results of the distant view and the close view, determines the position of the bridge and the position of the camera, and outputs the determination result to the image processing result memory unit 205. For example, in the case of the comparison results of FIGS. 5 to 7, the product A1 ′ (= 300 [mm]) of the distant view displacement amount A1 (= 10 [pix]) and the coefficient α (= 30 [mm / pix]) in FIG. ) And the product A2 ′ (= 400 [mm]) of the displacement A2 (= 20 [pix]) and the coefficient β (= 20 [mm / pix]) of the foreground in FIG. To calculate the actual displacement difference A3 ′ between the distant view and the close view.

 Further, the displacement angles θ1 and θ2 are also compared. Here, the coefficients α and β are coefficients for correcting the influence of the distance from the camera on the distance, and values are set according to the distance from the camera. The coefficients α and β can be calculated by multiplying the displacement amounts A1 and A2 with the actual displacement amounts A1 ′ and A2 ′, respectively. From the displacement amount A1 ′ of the distant view image, it is determined that the camera is displaced 300 mm in the right direction, and from the displacement difference A3 ′, it is determined that the bridge is displaced 100 mm in the left direction (FIG. 8). The displacement angles θ1 and θ2 are determined to be the same.

 FIG. 9 is a processing flow of the structure abnormality determination system according to the present embodiment. In step S1, background image data at the time is created from the input image data of the monitoring camera 4 (corresponding to the operation of the background image creation unit 202 in FIG. 3). In step S2, the displacement of the surveillance camera 4 and the displacement of the structure are determined from the difference between the background image at that time and the latest background image (corresponding to the operation of the image processing unit 204 in FIG. 4). In step S3, the displacement of the monitoring camera 4, the displacement of the structure, and the system abnormality are determined from the determination result of the image processing unit 204 and the measurement value of the displacement meter 1 (corresponding to the operation of the abnormality determination unit 207 in FIG. 2). .

 FIG. 10 is a processing flow showing the detailed operation contents at the time of creating a background image (corresponding to step S1 of the processing flow of FIG. 9) of the structure abnormality determination system according to the present embodiment. In step S101, the image data captured by the monitoring camera 4 is acquired, and the averaged image data stored in the averaged image memory unit 202b is acquired.

 In step S102, the images are averaged from the captured image and the averaged image acquired in step S101, and the elapsed time from the start of the averaging of the image is stored in the averaging time memory unit 202b. In step S103, it is determined whether the elapsed time stored in the averaging time memory unit 202b has reached a predetermined fixed time. If the fixed time has been reached, the process proceeds to step S104. If the fixed time has not been reached, the process proceeds to step S105.

 In step S104, the averaged image data is stored in the background image storage unit 203, and the averaged image data stored in the averaged image memory unit 202b and the elapsed time data stored in the averaged time memory unit are deleted. To end the process. In step S105, the averaged image data is stored in the averaging time memory unit, and the process ends.

 FIG. 11 is a processing flow showing detailed operation contents during image processing (corresponding to step S2 of the processing flow of FIG. 9) of the structure abnormality determination system according to the present embodiment. In step S201, the distant view image and the foreground image are divided with respect to the most recent background image X and the background image Y of the current time by mask processing. In step S202, the difference between the distant view image Yf of the background image at that time and the distant view image Xf of the latest background image is extracted. That is, the distant view image Xf is subtracted from the distant view image Xf, and the number of pixels having a luminance difference of a certain level or more is counted.

 In step S203, it is determined from the difference extraction result whether the number of pixels having a certain luminance difference or more is equal to or greater than a threshold value. If the determination result is greater than or equal to the threshold value, the process proceeds to step S204. If the determination result is not greater than or equal to the threshold value, the process proceeds to step S209. In step S204, the displacement amount A1 and the displacement angle θ1 of the distant view images Xf and Yf are calculated by tracking the corresponding points. Here, the displacement amount acquired by the image processing unit 204 is an approximate value for seeing the displacement difference between the distant view and the near view, and the accurate displacement difference is acquired by the abnormality determination unit 207.

 In step S205, the displacement amount A2 and the displacement angle θ2 of the foreground images Xc and Yc are calculated by tracking the corresponding points. In step S206, it is determined whether α × A1 = β × A2 (1) for the coefficients α and β and the displacement amounts A1 and A2, and θ1 = θ2 (2) for the displacement angles θ1 and θ2. . When both expressions (1) and (2) hold, the process proceeds to step S207, and when either expression (1) or (2) does not hold, the process proceeds to step S208. In step S207, it is determined that the structure is not displaced but the camera is displaced, and the process is terminated. In step S208, it is determined that both the structure and the camera are displaced, and the process ends.

 In step S209, the difference between the foreground image Yc of the background image at the time and the foreground image Xc of the latest background image is extracted. That is, the foreground image Yc is subtracted from the foreground image Xc, and the number of pixels having a luminance difference of a certain level or more is counted. In step S210, it is determined from the difference extraction result whether the number of pixels having a certain luminance difference or more is equal to or greater than a threshold value. If the determination result is greater than or equal to the threshold value, the process proceeds to step S211. If the determination result is not equal to or greater than the threshold value, the process proceeds to step S212. In step S212, it is determined that the structure is not displaced, and the process ends. In step S208, it is determined that the structure is displaced, and the process ends.

 12 to 15 are process flows showing detailed operation contents at the time of abnormality determination (corresponding to step S3 of the process flow of FIG. 9) of the structure abnormality determination system according to the present embodiment. In step S301, the determination result in the image processing unit 204 is acquired. In step S302, it is determined whether there is a displacement in the structure based on the determination result in step S301. If there is a displacement, the process proceeds to step S303, and if there is no displacement, the process proceeds to step S305.

 In step S303, measurement information is acquired by the displacement meter 1. In step S304, it is determined from the measurement information obtained by the displacement meter 1 whether there is a displacement in the structure. If there is a displacement, the process proceeds to step S311 in FIG. 13, and if there is no displacement, the process proceeds to step S321 in FIG. In step S305, measurement information is acquired by the displacement meter 1.

 In step S306, it is determined from the measurement information obtained by the displacement meter 1 whether there is a displacement in the structure. If there is a displacement, the process proceeds to step S331 in FIG. 15, and if there is no displacement, the process proceeds to S307. In S307, it is determined that the structure is in a normal state with no displacement, and the process ends. However, when it is determined that the camera is displaced at the stage of processing of the image processing unit 204, a display indicating that “system (camera) abnormality may be present” is displayed.

 In step S311 of FIG. 13, the displacement amount and the displacement angle are determined by tracking corresponding points of the image. In step S312, the data obtained in step S311 and the detected value of the displacement meter 1 are matched. In step S313, it is determined whether or not the results of both data in step S311 substantially match. If they match, the process proceeds to step S314, and if they do not match, the process proceeds to step S315. In step S314, it is determined that a displacement has occurred in the bridge girder 14, and the process ends. In step S315, it is determined that there is a possibility of system abnormality.

 In step S321 in FIG. 14, the displacement amount and the displacement angle are determined by tracking the corresponding points in the image. In step S322, it is determined whether the displacement direction / displacement amount has been detected as a result of tracking the corresponding points of the image. If it can be detected, the process proceeds to step S323, and if it cannot be detected, the process proceeds to step S324. In step S323, it determines with the displacement of the bridge girder 14 having generate | occur | produced, and complete | finishes a process.

 In step S324, it is determined whether a change in the road surface portion is detected as a result of tracking the corresponding points in the image. If there is a change in the road surface portion, the process proceeds to step S325, and if no change in the road surface portion is detected, the process proceeds to step S326. In step S325, it is determined that the change is caused by a road surface condition such as a traffic jam on the road or a fallen object on the road, and the process ends. In step S326, it is determined that there is a possibility of system abnormality such as a camera.

 In step S331 in FIG. 15, it is determined whether the detection result of the displacement meter 1 is equal to or less than a specified value. Here, the specified value is a value corresponding to a limit value at which the displacement amount clearly changes on the image. In step S332, if the determination result is equal to or less than the specified value, the process proceeds to step S333, and if not, the process proceeds to step S334. In step S333, it is determined that the bridge girder 14 having a small displacement is generated, and the process is terminated. In step S334, it is determined that there is a possibility of system abnormality.

 Note that the structure abnormality determination system according to the present embodiment may be realized by dedicated hardware, or may be configured by a computer system such as a personal computer and realize each function of the system shown in FIG. The function may be realized by executing a program for executing the program.

It is a lineblock diagram showing the whole structure abnormal structure discriminating system concerning the embodiment of the present invention. It is the functional block diagram which showed the process of the structure abnormality determination apparatus 2b in the structure abnormality determination system by embodiment of this invention, and the flow of data. It is a detailed functional block diagram of the background image preparation part in the structure abnormality determination apparatus 2b. It is a detailed functional block diagram of the image processing part in the structure abnormality determination apparatus 2b. It is a figure which shows an example of the processing method at the time of the image division in the image processing part of embodiment of this invention. It is a figure which shows an example of the processing method of the distant view image in the image processing part of embodiment of this invention. It is a figure which shows an example of the processing method of the foreground image in the image processing part of embodiment of this invention. It is a figure which shows an example of the process result in the image process part of embodiment of this invention. It is a processing flow which shows the whole operation | movement content of the structure abnormality determination system by this embodiment. It is a processing flow which shows the detailed operation | movement content at the time of the background image preparation of the structure abnormality determination system by this embodiment. It is a processing flow which shows the detailed operation | movement content at the time of the image processing of the structure abnormality determination system by this embodiment. It is a processing flow at the time of structure abnormality determination of the structure abnormality determination system concerning embodiment of this invention. It is a processing flow when it is determined that there is a displacement in the structure in both the photographed image and the displacement meter result in the processing at the time of determining the structure abnormality. It is a processing flow when it is determined that there is a displacement in the structure only in the result of the captured image in the processing at the time of determining the structure abnormality. It is a processing flow when it is determined that there is a displacement in the structure only in the result of the displacement meter in the process at the time of determining the structure abnormality. It is a structural diagram showing a structure of a bridge for determining a structure abnormality according to an embodiment of the present invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Displacement meter, 2 ... Monitoring center, 3 ... Image transmission apparatus, 4 ... Surveillance camera, 2a ... Analyzer, 2b ... Structure abnormality determination apparatus

Claims (6)

Photographing means for photographing the structure and its surroundings;
A displacement measuring means installed in the structure;
A structure displacement determining means for determining a displacement state of the structure from the image data photographed by the photographing means and the displacement data measured by the displacement measuring means;
The structure displacement determining means includes
Image comparison determination means for comparing the image data for each time imaged by the imaging means to determine the displacement state of the structure;
An abnormality determination unit that determines a displacement state of the structure and a system abnormality state from the determination result of the image comparison determination unit and the displacement data of the displacement measurement unit;
A structure abnormality discriminating system comprising:  The system according to claim 1, wherein the image data is averaged image data averaged for a predetermined time.  The image comparison means divides the image data into a distant view area and a foreground area, and compares the image data for each area for each time to determine the displacement state of the structure and the movement state of the photographing means. The structure abnormality discrimination system according to claim 1 or 2, characterized in that  The said image comparison determination means determines the displacement state of a structure by calculating the displacement difference in the predetermined corresponding point in the said image data for every time, The structure of any one of Claims 1-3 characterized by the above-mentioned. The structure abnormality discrimination system described in 1. A first step in which a photographing unit photographs a structure and a surrounding situation, and a displacement measuring unit measures displacement data of the structure;
A second step in which the image comparison determination means compares the image data for each time imaged by the imaging means to determine the displacement state of the structure;
A third step in which the abnormality determining means determines the displacement state of the structure and the state of the system abnormality from the determination result of the image comparison determining means and the displacement data of the displacement measuring means;
A structure abnormality discriminating method comprising: From the photographing means for photographing the structure and its surroundings, the displacement measuring means installed in the structure, the image data photographed by the photographing means, and the displacement data measured by the displacement measuring means A computer program for performing structure displacement determination processing in a structure abnormality determination system comprising structure displacement determination means for determining a displacement state of a structure,
An image comparison determination step for comparing the image data for each time imaged by the imaging means to determine the displacement state of the structure;
An abnormality determination step for determining the displacement state of the structure and the state of the system abnormality from the determination result in the image comparison determination step and the displacement data of the displacement measuring means,
A computer program for causing a computer to execute.

Images