JP2013167494A - Structure inspection support method, structure inspection support program, and structure inspection support apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a burden of an inspector when inspecting a road surface or the like of a bridge on site.SOLUTION: A moving image recording unit 22 acquires a moving image obtained by photographing an estimated damaged part that is estimated to be damaged in a road surface of a bridge and a vicinity of the estimated damaged part, with a photographing device 10. An inspection object image identification unit 24 accepts, from a consultant terminal 30, an instruction for identifying one frame (an inspection object image) in which the estimated damaged part is photographed in the plurality of frames included in the moving image. A mark image extraction unit 26 extracts a frame (a mark image) whose difference from the inspection object image is large, from frames photographed before/after a time when photographing the identified inspection object image. An inspection screen generating unit 28 generates an inspection screen for simultaneously displaying the mark image and the inspection object image.

Description

  This case relates to a structure inspection support method, a structure inspection support program, and a structure inspection support apparatus.

  Conventionally, an inspection support system has been proposed for smoothly performing inspection work on structures such as bridges. The main purpose of this inspection support system is to support simple inspections among various levels of inspection work such as simple inspections and detailed inspections of structures. Specifically, a conventional inspection support system performs a simple and smooth process of requesting an inspector (such as a construction consultant) to check a still image or a video of a structure photographed by a photographer (such as a local government employee). Is supporting.

  As a structure to be inspected, for example, there is a road surface of a bridge. Since the number of bridges to be inspected by one local government is very large, it is efficient to use the above-described inspection support system in inspecting the road surface of the bridge.

  Patent Document 1 discloses a road surface damage state inspection method that automatically detects a damaged portion of a road surface from an image taken by a camera while traveling on the road surface.

JP 2005-43324 A

  When using the above-described inspection support system, an inspector first refers to a captured image (still image or moving image) on the terminal, and specifies a damaged portion to be inspected in detail on the basis of the image. . Then, the inspector performs a detailed inspection of the road surface on the site while confirming the damaged portion to be inspected specified in advance on a portable terminal.

  In this case, since the road surface of the bridge is a wide flat surface with the same color and less irregularities, it may take a long time to find the damaged part on site.

  Note that the above-mentioned Patent Document 1 describes that the kilometer post that is closest to the automatically detected damaged portion is specified, and information about the kilopost is stored in the memory together with the image of the damaged portion. However, although the kilometer post is often installed on an expressway or the like, it is rarely installed on a general road or a bridge, so that it is difficult to use the kilometer post as a mark of a damaged portion of the road surface of the bridge.

  Structures other than bridges (river banks (slopes), retaining walls for preventing landslides, walls of buildings such as housing estates and buildings) are the same color and have a wide surface with few irregularities (or the same pattern). In addition, there are rare cases where landmarks such as kiloposts are installed.

  Therefore, the present case has been made in view of the above problems, and to provide a structure inspection support method, a structure inspection support program, and a structure inspection support device that can reduce the burden of on-site structure inspection. With the goal.

  As a result of analyzing the behavior of a photographer who photographs a structure such as a road surface of a bridge, the present inventors have found that the photographed person visually recognizes the damaged portion in order to easily identify the position of the damaged portion of the structure. After discovering in, I noticed that I often shoot around the damaged part before and after shooting the damaged part. The structure inspection support method, the structure inspection support program, and the structure inspection support apparatus described in this specification are based on such new knowledge.

  In the structure inspection support method described in this specification, an image group obtained by photographing an estimated damaged portion estimated to be damaged in a structure to be inspected and the vicinity of the estimated damaged portion with an imaging device. An acquisition step of acquiring, an accepting step of accepting an instruction specifying one image from which the estimated damaged portion is imaged among a plurality of images included in the image group, and a time at which the specified image was captured An extraction step for extracting at least one image having a difference from the specified image greater than or equal to a predetermined value from images taken before and after, and a screen for simultaneously displaying the extracted image and the specified image A structure inspection support method executed by a computer.

  The structure inspection support program described in this specification includes an image group obtained by photographing an estimated damaged portion estimated to be damaged in a structure to be inspected and the vicinity of the estimated damaged portion with a photographing apparatus. An image acquired and received before and after the time at which the specified image was captured, receiving an instruction specifying one image from which the estimated damaged portion was captured among a plurality of images included in the image group To extract at least one image whose difference from the specified image is greater than or equal to a predetermined value, and generate a screen on which the extracted image and the specified image are displayed simultaneously. This is a structure inspection support program.

  The structure inspection support device described in this specification is a group of images obtained by photographing an estimated damage portion estimated to be damaged in a structure to be inspected and the vicinity of the estimated damage portion with a photographing device. An acquisition unit for acquiring, an accepting unit for receiving an instruction specifying one image from which the estimated damaged portion is captured among a plurality of images included in the image group, and a time at which the specified image is captured An extraction unit that extracts at least one image having a difference from the specified image greater than or equal to a predetermined value from images taken before and after, and a screen that simultaneously displays the extracted image and the specified image And a generating unit for generating.

  The structure inspection support method, the structure inspection support program, and the structure inspection support device described in this specification have an effect of reducing the burden of structure inspection on site.

It is a figure showing roughly the composition of the bridge inspection support system concerning one embodiment. It is a hardware block diagram of a server. It is a functional block diagram of a bridge inspection support system. It is a figure which shows map DB. It is a figure which shows animation definition DB. FIG. 6A is a diagram illustrating the moving image DB, FIG. 6B is a diagram illustrating the inspection target image position table, and FIG. 6C is a diagram illustrating the setting DB. It is a figure which shows mark image DB. It is a flowchart which shows the process of a moving image recording part. It is a figure which shows an example of a bridge name input screen. It is a flowchart (the 1) which shows an inspection object image specific process. It is a flowchart (the 2) which shows an inspection object image specific process. It is a figure which shows an example of a list screen. It is a figure which shows an example of a moving image file reproduction | regeneration screen. It is a flowchart (the 1) which shows the process of a mark image extraction part. It is a flowchart (the 2) which shows the process of a mark image extraction part. It is a flowchart (the 1) which shows the screen display process for an inspection. It is a flowchart (the 2) which shows the screen display process for inspection. It is a flowchart (the 3) which shows the screen display process for inspection. It is a figure (the 1) which shows the screen for an inspection. It is a figure (the 2) which shows the screen for an inspection. It is a figure which shows an example of the screen which can be displayed with the screen of FIG. 19, FIG.

  Hereinafter, an embodiment will be described in detail with reference to FIGS. FIG. 1 schematically shows a bridge inspection support system 100 including a server 20 as a structure inspection support apparatus.

  As shown in FIG. 1, the bridge inspection support system 100 includes an imaging device 10, a server 20, a consultant terminal 30, and an inspection terminal 50. Each device (10, 20, 30, 50) of the bridge inspection support system 100 is connected to a network 40 such as the Internet.

  The imaging device 10 is, for example, a video camera capable of shooting a moving image (for example, high-definition shooting). A moving image can be said to be an aggregate (image group) of a plurality of frames (images). As illustrated in FIG. 3, the imaging device 10 includes an imaging unit 14, a display unit 15, an input unit 16, a position acquisition unit 12, an angular velocity sensor 17, and a communication unit 18.

  The photographing unit 14 photographs a road surface of a bridge to be inspected with a moving image based on an instruction input by a photographer (for example, a local government staff) via the input unit 16. The display unit 15 includes a liquid crystal display and the like, displays various menu screens, and displays a moving image (finder image) being shot. The input unit 16 includes a touch panel and various buttons, receives an instruction from the photographer, and transmits the instruction to the imaging unit 14 and the communication unit 18. The position acquisition unit 12 includes a GPS module, and acquires information (latitude and longitude) related to the position of the imaging device 10. The angular velocity sensor 17 includes a gyro sensor and the like, and acquires information (gyro value) related to the posture of the photographing apparatus 10.

  The communication unit 18 includes a moving image file shot by the shooting unit 14, a time stamp when the shooting unit 14 shots a moving image, position information acquired by the position acquisition unit 12, information on an attitude acquired by the angular velocity sensor 17, and the like. Information including the information (hereinafter referred to as “photographing information”) is transmitted to the server 20.

Here, in this embodiment, a photographer who photographs using the photographing apparatus 10 captures a moving image under the following conditions based on a predetermined manual.
(1) The structure to be imaged is the road surface of a bridge.
(2) The photographer takes a picture of the road surface while checking the condition of the road surface with the naked eye.
(3) When the photographer confirms a portion that is estimated to be damaged (estimated damage portion) with the naked eye, the photographer photographs the periphery of the estimated damage portion before and after photographing the estimated damage portion. In this case, the photographer takes care to photograph a mark (a sign, an object having a characteristic shape, a color, dirt, etc.) that can be a mark around the estimated damaged portion.

  Returning to FIG. 1, the server 20 receives and holds the moving image file and the shooting information transmitted from the shooting device 10. Further, the server 20 generates a screen (moving image file playback screen (see FIG. 13)) for displaying (playing back) the moving image shot by the shooting device 10 in accordance with an instruction from the consultant terminal 30, and the consultant terminal To 30. Further, in response to an instruction from the consultant terminal 30, a screen (inspection screen (see FIGS. 19 and 20)) used when inspecting the estimated damaged portion of the road surface is generated on the site, and the inspection terminal 50 is inspected. Send.

  FIG. 2 shows the hardware configuration of the server 20. As shown in FIG. 2, the server 20 includes a CPU 90, a ROM 92, a RAM 94, a storage unit (here, HDD (Hard Disk Drive)) 96, a network interface 97, a portable storage medium drive 99, and the like. Each component of the 20 components is connected to a bus 98. In the server 20, the CPU 90 executes a program (structure inspection support program) stored in the ROM 92 or the HDD 96 or a program (structure inspection support program) read from the portable storage medium 91 by the portable storage medium drive 99. By doing so, the function of each part of FIG. 3 is implement | achieved. That is, in the server 20, the CPU 90 causes the moving image recording unit 22 as an acquisition unit, the inspection target image specifying unit 24 as a reception unit, the mark image extraction unit 26 as an extraction unit, and the inspection as a generation unit and a determination unit. The function of the screen generation unit 28 is realized. 3 also illustrates the map DB 61, the moving image definition DB 62, the moving image DB 63, the inspection target image position table 64, the setting DB 65, and the mark image DB 66 stored in the HDD 96 or the like.

  The moving image recording unit 22 refers to the map DB 61 and performs recording on the moving image definition DB 62 and the moving image DB 63 using the moving image file and the shooting information received from the communication unit 18. Here, the map DB 61 is a database for managing the location of place names, and as shown in FIG. 4, “place names” are defined by “latitude” and “longitude”.

  The moving image definition DB 62 is a database for managing information of moving image files captured by the image capturing device 10, and as shown in FIG. 5, “bridge name”, “shooting date”, “shooting location (shooting start point)”, Each field includes “moving image ID”, “shooting start position (latitude, longitude)”, “shooting end position (latitude, longitude)”, and “shooting direction”. In the fields of “shooting date”, “shooting start position (latitude, longitude)”, and “shooting end position (latitude, longitude)”, each piece of information included in the shooting information acquired from the shooting apparatus 10 is stored. In the “shooting location (shooting start point)” field, the position corresponding to the latitude and longitude recorded in the “shooting start position (latitude, longitude)” field is extracted from the map DB 61 and stored. In the “bridge name” field, the bridge name input from the input unit 16 of the photographing apparatus 10 is stored. In the “moving image ID” field, a unique character string assigned to the moving image file by the moving image recording unit 22 is stored. In the “shooting direction” field, a shooting direction derived from the contents of each field of “shooting start position (latitude, longitude)” and “shooting end position (latitude, longitude)” (16 directions indicated by numbers) Is stored.

  As shown in FIG. 6A, the moving image DB 63 has fields of “moving image ID” and “moving image file”.

  Returning to FIG. 3, in response to a request from the consultant terminal 30, the inspection target image specifying unit 24 generates a screen (moving image file reproduction screen (FIG. 13)) for reproducing the moving image stored in the moving image DB 63. The screen is transmitted to the consultant terminal 30. Further, the inspection target image specifying unit 24 specifies an image (a frame in which the estimated damaged portion is photographed) specified by the construction consultant on the moving image file reproduction screen as the inspection target image. The inspection target image specifying unit 24 stores the specification result in the inspection target image position table 64. Here, as shown in FIG. 6B, the inspection target image position table 64 has fields of “moving image ID”, “time stamp”, and “image snapshot”. The “image snapshot” field stores a snapshot of the frame specified as the inspection target image, and the “time stamp” field stores a time stamp corresponding to the snapshot (0 m. (Time set to 00s.00) is stored.

  Returning to FIG. 3, the mark image extraction unit 26 extracts the mark image from the moving image file with reference to the inspection object image position table 64 and the setting DB 65 according to the instruction from the inspection terminal 50, and extracts the mark image. Information is stored in the mark image DB 66. Here, the mark image refers to an image around the estimated damaged part (based on a sign, an object having a characteristic shape or color, dirt, etc.) taken based on the manual (3) described above. Image). The mark image extraction method performed by the mark image extraction unit 26 will be described later.

  As shown in FIG. 6C, the setting DB 65 defines a setting value (2 minutes in FIG. 6C) that sets a range for extracting the mark image. As shown in FIG. 7, the mark image DB 66 includes fields of “moving image ID”, “time stamp”, “mark image file”, “shooting position (latitude, longitude)”, “depth information”, and “left / right information”. Have The “time stamp” field stores the time stamp of the inspection target image, and the “mark image file” field stores a snapshot of the mark image. In addition, in “shooting position (latitude, longitude)”, the shooting position of the mark image is input, and in the “depth information” and “left / right information” fields, information on the shooting position of the mark image with respect to the shooting position of the inspection target image is input. Is stored.

  Returning to FIG. 3, the inspection screen generator 28 generates an inspection screen (FIG. 19, etc.) based on the mark image DB 66 in response to a request from the inspection terminal 50. In addition, the inspection screen generation unit 28 transmits the generated inspection screen to the inspection terminal 50.

  The consultant terminal 30 is a terminal used by a user (such as a construction consultant) who checks the road surface of the bridge on the screen. The consultant terminal 30 displays a screen (moving image file reproduction screen (FIG. 13)) provided from the server 20. The construction consultant uses the moving image file reproduction screen (FIG. 13) displayed on the consultant terminal 30 to view the captured moving image file and inspect the road surface of the bridge. In addition, the construction consultant operates the consultant terminal 30 as necessary to specify a frame (inspection target image) in which a portion (estimated damage portion) estimated to be damaged on the road surface is captured. To do.

  As shown in FIG. 3, the consultant terminal 30 includes a control unit 32, a display unit 34, and an input unit 36. The control unit 32 includes a CPU, a ROM, a RAM, and the like, and comprehensively controls each unit of the consultant terminal 30 and transmits and receives data and the like (screen and instructions) to and from the server 20. The display unit 34 includes a liquid crystal display and the like, and displays a screen received from the server 20 under the instruction of the control unit 32. The input unit 36 receives an instruction input from the construction consultant and transmits it to the server 20 via the control unit 32.

  The inspection terminal 50 is a portable terminal used by a local inspector (for example, a construction consultant) who confirms the road surface of the bridge in detail on site. The inspection terminal 50 displays a screen (inspection screen (FIG. 19)) provided from the server 20. As shown in FIG. 3, the inspection terminal 50 includes a control unit 52, a display unit 54, an input unit 56, and a position acquisition unit 58. The control unit 52 includes a CPU, a ROM, a RAM, and the like, and comprehensively controls each unit of the inspection terminal 50 and transmits and receives data and the like (screen and instructions) to and from the server 20. The display unit 54 includes a liquid crystal display or the like, and displays a screen received from the server 20 under the instruction of the control unit 52. The input unit 56 receives an instruction input from the field inspector and transmits the instruction to the server 20 via the control unit 52. The position acquisition unit 58 includes a GPS module, and acquires information (latitude, longitude) regarding the position of the inspection terminal 50. The field inspector finds the estimated damaged portion while looking at the inspection screen, and performs a detailed inspection of the estimated damaged portion. The person using the inspection terminal 50 and the consultant terminal 30 may be different persons or the same person.

  Next, regarding the processing in this embodiment ((A) processing of the moving image recording unit 22, (B) inspection target image specifying processing, (C) processing of the mark image extraction unit 26, (D) inspection screen display processing) This will be described in detail.

(A) Processing of Moving Image Recording Unit 22 First, the processing of the moving image recording unit 22 will be described in detail along the flowchart of FIG.

  FIG. 8 shows a process of the moving image recording unit 22 of the server 20 in a flowchart. The processing in FIG. 8 is performed, for example, from the time when the photographer completes photographing of the road surface of the bridge by the photographing device 10 and inputs an instruction to transmit the photographed moving image to the server 20 via the input unit 16. Shall be started.

  In the process of FIG. 8, first, in step S <b> 10, the moving image recording unit 22 displays a bridge name input screen on the display unit 15 of the photographing apparatus 10. The bridge name input screen is, for example, a screen as shown in FIG. In the bridge name input screen of FIG. 9, the photographer selects the bridge name after narrowing down by the prefecture name, and transmits the bridge name to the server 20 by pressing the send button after selection. Can be done. In this case, the server 20 stores a bridge name DB that stores the prefecture name and the bridge name, and generates the screen of FIG. 9 using the bridge name DB. However, the present invention is not limited to this, and the moving image recording unit 22 may display a bridge name input screen or the like on which the bridge name can be directly input on the display unit 15.

  Next, in step S12, the moving image recording unit 22 stands by until a bridge name is received. In other words, the moving image recording unit 22 makes a positive determination in step S12 when the photographer inputs a bridge name in the photographing apparatus 10 (step in which the transmission button in FIG. 9 is pressed), and proceeds to step S14.

  If transfering it to step S14, the moving image recording part 22 will record the accepted bridge name in moving image definition DB62 (FIG. 4). Next, in step S <b> 16, the moving image recording unit 22 downloads a moving image file from the photographing apparatus 10. At the time of downloading a moving image file, the moving image recording unit 22 captures shooting information of the moving image file (information such as shooting date information, shooting start position, shooting end position, and information about the direction of the shooting apparatus 10 (gyro value)). Also get.

  When the downloading of the moving image file in step S16 is completed, in the next step S18, the moving image recording unit 22 allocates a moving image ID to the moving image file and records it in the moving image DB 63 in association with the file name. Next, in step S <b> 20, the moving image recording unit 22 records the moving image ID in the moving image definition DB 62.

  Next, in step S <b> 22, the moving image recording unit 22 acquires shooting date information from the shooting information and records it in the moving image definition DB 62. Next, in step S <b> 24, the moving image recording unit 22 acquires shooting start position (latitude, longitude) information from the shooting information and records it in the “shooting start position” field of the moving image definition DB 62. Next, in step S <b> 26, the moving image recording unit 22 acquires shooting end position (latitude, longitude) information from the shooting information and records it in the “shooting end position” field of the moving image definition DB 62.

  Next, in step S28, the moving picture recording unit 22 acquires the place name of the shooting start position from the map DB 61 based on the shooting start position (latitude, longitude), and sets the “shooting place (shooting start point)” in the moving picture definition DB 62. Record in the field.

  Next, in step S <b> 30, the moving image recording unit 22 calculates the azimuth using the shooting start position and the shooting end position of the moving image definition DB 62 and records them in the “shooting direction” field of the moving image definition DB 62.

  Through the above processing, the moving image recording unit 22 can record the acquired moving image file information in each of the moving image definition DB 62 and the moving image DB 63.

  In the above description, the case where the moving image file and the shooting information are transmitted from the shooting device 10 to the server 20 via the network 40 has been described. However, the present invention is not limited to this. For example, a moving image file or shooting information shot by the shooting device 10 may be delivered to the server 20 via a storage medium (such as an optical disk or a flash memory). In the case of using a storage medium, the photographing apparatus 10 may not have the communication unit 18 (not necessarily connected to the network 40). Alternatively, the photographing apparatus 10 may be connected to a terminal such as a PC, and a moving image file or photographing information may be transmitted from the terminal to the server 20.

(B) Inspection target image specifying process Next, the inspection target image specifying process will be described in detail with reference to the flowcharts of FIGS. 10 and 11. The inspection target image specifying process is realized by simultaneous parallel processing of the inspection target image specifying unit 24 of the server 20 and the control unit 32 of the consultant terminal 30. In FIG. 10 and FIG. 11, each processing and determination are arranged in consideration of the processing timing of the server 20 and the processing timing of the consultant terminal 30.

  The process of FIG. 10 is started from a stage where a construction consultant using the consultant terminal 30 issues a moving image reproduction request using the input unit 36. In the process of FIG. 10, first, in step S200, the control unit 32 transmits a moving image reproduction request to the server 20 (inspection target image specifying unit 24).

  On the other hand, on the server 20 side, the inspection target image specifying unit 24 stands by until a moving image reproduction request is received from the consultant terminal 30 in step S100. Therefore, the inspection target image specifying unit 24 proceeds to step S102 when the control unit 32 performs the process of step S200. Then, in step S102, the inspection target image specifying unit 24 acquires all bridge names registered from the moving image definition DB 62, creates a list screen, and transmits the list screen to the consultant terminal 30. Note that the list screen is, for example, a screen as shown in FIG. The list screen in FIG. 12 includes a list of bridge names and a transmission button that is pressed after the construction consultant selects one bridge name from the list of bridge names.

  On the other hand, on the consultant terminal 30 side, the control unit 32 stands by after step S200 until a list screen is received from the server 20 in step S202. Therefore, the control part 32 will transfer to step S204 in the stage in which the process of step S102 was performed by the server 20 side. In step S204, the control unit 32 displays the list screen (FIG. 12) received from the server 20 on the display unit 34.

  Next, in step S206, the control unit 32 stands by until accepting selection of a bridge name. When the construction consultant selects a bridge name on the list screen of FIG. 12 and presses the transmission button, the control unit 32 proceeds to step S208. If transfering it to step S208, the control part 32 will transmit the bridge name selected by the construction consultant to the server 20.

  On the other hand, on the server 20 side, the inspection target image specifying unit 24 stands by after step S102 until it receives a bridge name in step S104. Therefore, the inspection target image specifying unit 24 proceeds to step S106 after the process of step S208 is performed on the consultant terminal 30 side.

  If transfering it to step S106, the inspection object image specific | specification part 24 will acquire moving image ID corresponding to the received bridge name from moving image definition DB62. Next, in step S108, a moving image file corresponding to the moving image ID is acquired from the moving image DB 63, and a moving image file reproduction screen (see FIG. 13) is generated and transmitted to the consultant terminal 30. Note that the movie file playback screen in FIG. 13 includes a playback screen, operation buttons for operating the playback screen (play button, pause button, fast forward button, rewind button, etc.), and “check required screen”. And a check button for specifying. Thereafter, the process proceeds to step S110 in FIG.

  On the other hand, on the consultant terminal 30 side, the control unit 32 stands by after step S208 until the moving image file reproduction screen is received from the server 20 in step S210. Therefore, the control unit 32 proceeds to step S212 when the moving image file playback screen is transmitted from the server 20 side in step S108.

  In step S212, the control unit 32 displays the moving image file reproduction screen shown in FIG.

  Next, the control unit 32 reproduces the moving image file on the moving image file reproduction screen in step S214 of FIG.

  Next, in step S216, the control unit 32 determines whether or not the check button has been pressed by the construction consultant. Here, when the construction consultant finds an image (frame) in which a portion (estimated damaged portion) that is estimated to be damaged during the reproduction of the moving image is detected, the reproduction of the moving image is temporarily stopped. The check button shown in Fig. 2 is pressed (an instruction specifying the image of the estimated damaged portion is issued). As described above, when the construction consultant gives an instruction specifying the image of the estimated damaged portion, the determination in step S216 is affirmed, and the process proceeds to step S218. On the other hand, if the determination in step S216 is negative, the process proceeds to step S224.

  When the determination in step S216 is affirmed and the process proceeds to step S218, the control unit 32 acquires an image image (snapshot) being reproduced. Next, in step S220, the control unit 32 acquires a time stamp from the start of reproduction of the acquired snapshot. Next, in step S <b> 222, the control unit 32 transmits an image image (snapshot) and a time stamp to the server 20. Thereafter, the process proceeds to step S224, and the processes and determinations in steps S216 to S224 are repeated until the construction consultant finishes the regeneration.

  On the other hand, on the server 20 side, the inspection target image specifying unit 24 stands by after step S108 until receiving a time stamp and a snapshot in step S110. Therefore, the inspection target image specifying unit 24 proceeds to step S112 after the process of step S222 is performed on the consultant terminal 30 side.

  In step S112, the inspection target image specifying unit 24 records the received time stamp in the inspection target image position table 64 (FIG. 6B). Thereafter, the inspection target image specifying unit 24 determines whether or not the reproduction end information has been received from the consultant terminal 30 in step S114. If the determination here is negative, the process returns to step S110.

  On the other hand, when the construction consultant inputs the end of regeneration at the consultant terminal 30, the determination in step S224 is affirmed, and the control unit 32 proceeds to step S226. Then, in step S226, the control unit 32 transmits the reproduction end information to the server 20, and ends all the processes in FIGS. On the other hand, when the process of step S226 is executed, the determination of step S114 is affirmed on the server 20 (inspection target image specifying unit 24) side, and all the processes of FIGS. 10 and 11 are terminated.

  As a result of the processing as described above, the result of the simple inspection by the construction consultant using the moving image (information on the image of the estimated damaged part (inspection target image) specified by the construction consultant) is used as the inspection target image position table 64. Can be recorded.

(C) Processing of Mark Image Extraction Unit 26 Next, the processing of the mark image extraction unit 26 will be described in detail with reference to the flowcharts of FIGS.

  In the process of FIG. 14, first, in step S300, the mark image extraction unit 26 waits until it receives a bridge name that is a mark image creation target. In this case, the mark image extraction unit 26 proceeds to step S302 when the site inspector inputs the name of the bridge to be inspected from the input unit 56 of the inspection terminal 50.

  Next, in step S302, the mark image extraction unit 26 acquires the moving image ID corresponding to the accepted bridge name from the moving image definition DB 62. Next, in step S304, the mark image extraction unit 26 refers to the inspection target image position table 64 corresponding to the acquired moving image ID, and acquires all time stamps corresponding to the moving image ID.

  Next, in step S <b> 306, the mark image extraction unit 26 acquires a setting value from the setting DB 65. In this embodiment, as an example, as shown in FIG. 6C, the set value is 2 minutes.

  Next, in step S308, the mark image extraction unit 26 uses an image (inspection target image) corresponding to the time stamp recorded in the inspection target image position table 64 by the inspection target image specifying unit 24 in step S112 described above as a starting image. Images for a predetermined value (2 minutes) before and after are acquired. Next, in step S310, the mark image extraction unit 26 compares the starting image with the preceding and succeeding images (frames) within a predetermined value (2 minutes), and greatly differs (the difference is greater than or equal to a predetermined value). Take multiple snapshots.

  Next, in step S312, the mark image extraction unit 26 selects a snapshot having the same focal length as that before and after (the zoom is fixed) from the plurality of snapshots acquired in step S310. 1 (2 in total). Note that the two snapshots acquired in step S312 are mark images. In addition, as the mark image, an image (for example, in the inspection screen of FIG. 19) in which an object that can be a mark around the estimated damaged portion (a sign, an object having a characteristic shape or color, dirt, etc.) is photographed. It is considered that the mark image immediately after (left) and the previous mark image (right) displayed on the screen are acquired.

  Next, in step S314, the mark image extraction unit 26 records the acquired snapshot and the time stamp of the origin image in the “mark image file” and “time stamp” fields of the mark image DB 66. After step S314, the process proceeds to step S320 in FIG.

  When the process proceeds to step S320 in FIG. 15, the mark image extraction unit 26 compares the time stamp of the mark image acquired in step S312 with the time stamp of the starting image, and the time stamp of the mark image is greater than the time stamp of the starting image. If it is before, “near” is recorded in the “depth information” field of the mark image DB 66. If the time stamp of the mark image is later than the time stamp of the starting image, “back” is recorded in the “depth information” field of the mark image DB 66.

  Next, in step S322, the mark image extraction unit 26 acquires position information of the photographing apparatus 10 when each mark image is photographed from the photographing information, and records it in the mark image DB 66. Next, in step S324, the mark image extraction unit 26 acquires the gyro value of the photographing apparatus 10 when each mark image is photographed from the photographing information.

  Next, in step S326, the mark image extraction unit 26 determines whether the mark image is located on the left or right of the starting point image (inspection target image) based on the gyro value. Next, in step S328, the mark image extraction unit 26 records the determination result (“left” or “right”) in step S326 in the “left / right information” field of the mark image DB 66.

  Next, in step S330, the mark image extraction unit 26 determines whether or not there is a next time stamp (whether or not there is a time stamp that has not been processed in steps S306 to S328 among the time stamps acquired in step S304). to decide. When judgment here is affirmed, it returns to step S306. On the other hand, if the determination in step S330 is negative, all the processes in FIGS. 14 and 15 are terminated.

  By performing the processing as described above, information on the mark image corresponding to each starting image can be recorded in the mark image DB 66.

  14 and 15 can also be performed by batch processing. In this case, the processes shown in FIGS. 14 and 15 may be performed on all image snapshots (inspection target images) newly registered in the inspection target image position table 64 between the batch processes. .

(D) Screen display process for inspection Next, the screen display process for inspection will be described in detail with reference to the flowcharts of FIGS. In FIG. 16 to FIG. 18, each process and determination are arranged in consideration of the process timing of the server 20 and the process timing of the inspection terminal 50. The process of FIGS. 16 to 18 is started from the stage where the local inspector inputs an inspection screen display request from the input unit 56 of the inspection terminal 50 when the local inspector arrives at the site and starts the inspection. Is done.

  First, in step S500 in FIG. 16, the control unit 52 transmits a display request for an inspection screen to the server 20. On the other hand, since the inspection screen generation unit 28 is on standby until a display request for the inspection screen is received in step S400, step S402 is performed when the processing of step S500 is performed in the inspection terminal 50. Migrate to

  In step S402, the inspection screen generation unit 28 acquires the bridge name from the moving image definition DB 62, creates a list screen (similar to FIG. 12), and transmits it to the inspection terminal 50. On the other hand, the control unit 52 of the inspection terminal 50 stands by until a list screen is received in step S502. Therefore, the control unit 52 proceeds to step S504 when the process of step S402 is performed in the inspection screen generation unit 28. In step S504, the control unit 52 displays a list screen on the display unit 54.

  Next, in step S <b> 506, the control unit 52 acquires position information of the inspection terminal 50 (that is, a local inspector) from the position acquisition unit 58. Next, in step S508, the control unit 52 waits until accepting selection of a bridge name. Here, when the local inspector selects a bridge name to be inspected from the list screen, the control unit 52 proceeds to step S510. In step S <b> 510, the control unit 52 transmits the bridge name and position information to the server 20. Thereafter, the process proceeds to step S512 in FIG.

  On the other hand, on the server 20 side, the inspection screen generation unit 28 stands by after step S402 until it receives the bridge name and position information in step S404. Accordingly, at the stage where the process of step S510 is performed on the inspection terminal 50 side, the inspection screen generation unit 28 proceeds to step S406.

  In step S406, the inspection screen generation unit 28 acquires the moving image ID, the shooting start position, and the shooting end position corresponding to the received bridge name from the moving image definition DB 62. Next, in step S410, the inspection screen generation unit 28 acquires all the time stamps corresponding to the moving image ID from the inspection target image position table 64. Next, in step S412, the inspection screen generation unit 28 determines whether the received position information is closer to the imaging start position or the imaging end position. Thereafter, the process proceeds to step S414 in FIG.

  When the process proceeds to step S414 in FIG. 17, the inspection screen generation unit 28 determines whether the received position information is close to the photographing end position as a result of the determination in step S412. That is, in step S412, the inspection screen generation unit 28 checks whether the field inspector is inspecting the road surface from the same direction as the direction photographed by the photographing apparatus 10, or is opposite to the direction photographed by the photographing apparatus 10. From the direction, determine whether the local inspector will inspect the road surface. If the determination here is negative, that is, if the direction of inspecting the road surface of the bridge is the same as the shooting direction, the process proceeds to step S416. In step S416, the inspection screen generation unit 28 acquires, from the inspection target image position table 64, a snapshot corresponding to the earliest time stamp among all the time stamps acquired in step S410. In step S418, the inspection screen generation unit 28 acquires a mark image corresponding to the time stamp acquired in step S410 from the mark image DB 66. Thereafter, in step S430, the inspection screen generation unit 28 displays a predetermined depth diagram on the inspection screen and embeds the mark image position, inspection image position, and current position information. In step S432, the acquired snapshot is displayed. The mark image is inserted into a predetermined position on the inspection screen. As a result of steps S430 and S432, an inspection screen as shown in FIG. 19 is generated.

  On the other hand, if the determination in step S414 is affirmative, that is, if the direction in which the road surface of the bridge is inspected is opposite to the imaging direction, the process proceeds to step S420. In step S420, the inspection screen generation unit 28 acquires a snapshot corresponding to the latest time stamp from all the time stamps acquired in step S410 from the inspection target image position table 64. Next, in step S422, the inspection screen generation unit 28 acquires a mark image corresponding to the acquired time stamp from the mark image DB 66.

  Next, in step S424, the inspection screen generation unit 28 replaces each time stamp with a time stamp from the photographing end time. Next, in step S426, the inspection screen generation unit 28 flips the snapshot upside down. Next, in step S428, the inspection screen generation unit 28 reverses the depth information and left / right information of the mark image.

  Next, in step S430, the inspection screen generation unit 28 displays a predetermined depth diagram on the inspection screen and embeds the mark image position, inspection image position, and current position information. In step S432, the inspection screen generation unit 28 inserts the snapshot (upside down) and the mark image at a predetermined position on the inspection screen. As a result of these steps S430 and S432, for example, even when the same estimated damage portion as that in FIG. 19 is inspected, the inspection is performed from the direction opposite to the photographing direction, and therefore, as shown in FIG. An inspection screen is generated.

  After the inspection screen of FIG. 19 or FIG. 20 is generated, when the process proceeds to step S434, the inspection screen generation unit 28 transmits the inspection screen to the inspection terminal 50. Thereafter, the process proceeds to step S436 in FIG.

  On the other hand, on the inspection terminal 50 side, the control unit 52 stands by until an inspection screen is received in step S512 after the process of step S510. Therefore, at the stage where the process of step S434 is performed on the server 20 side, the control unit 52 proceeds to step S514. If transfering it to step S514, the control part 52 will display the screen for an inspection (FIG. 19 or FIG. 20) on the display part 54. FIG. Thereafter, the process proceeds to step S516 in FIG.

  When the process proceeds to step S516 in FIG. 18, the control unit 52 determines whether or not position information has been acquired from the position acquisition unit 58. If the determination here is affirmative, the process proceeds to step S518, and the control unit 52 transmits the position information to the server 20, and returns to step S512 in FIG. On the other hand, if the determination in step S516 is negative, the process proceeds to step S520, and the control unit 52 determines whether or not the next display request has been received from the local inspector. In this case, when the local inspector presses the “next” button on the inspection screen of FIG. 19 or FIG. 20, the determination in step S520 is affirmed, and the process proceeds to step S522. In this case, the control unit 52 transmits the next display request to the server in step S522, and then returns to step S512. On the other hand, if the “next” button has not been pressed, the determination in step S520 is denied and the process proceeds to step S524.

  In step S524, the control unit 52 determines whether a display end request has been received. If the determination is negative, the control unit 52 returns to step S516. If the determination is positive, The process moves to S526. When the process proceeds to step S526, the control unit 52 transmits a display end request to the server 20 and ends all the processes in FIGS.

  On the other hand, on the server 20 side, after step S434 in FIG. 17, in step S436, it is determined whether or not the inspection screen generation unit 28 has received the position information from the inspection terminal 50. When the determination here is affirmative (when the process of step S518 is performed in the inspection terminal 50), the process proceeds to step S438, and the inspection screen generation unit 28 displays the inspection screen (FIG. 19 or FIG. 19). The current position information of 20) is changed, and a new inspection screen is transmitted to the inspection terminal 50. Thereafter, the process proceeds to step S440. In this case, the control unit 52 of the inspection terminal 50 affirms the determination in step S512 of FIG. 17, so that a new inspection screen is displayed on the display unit 54 in step S514.

  On the other hand, when the determination in step S436 is negative, or after step S438, in step S440, the inspection screen generation unit 28 determines whether or not the next display request has been received from the inspection terminal 50. If the determination here is affirmative (when the process of step S522 is performed in the inspection terminal 50), in step S442, the inspection screen generation unit 28 determines whether there is a next time stamp (step S42). It is determined whether or not the processing related to all the time stamps acquired in S410 has been completed. If the determination here is affirmed, the process returns to step S414 in FIG. In addition, when step S416 and step S420 are performed after the second time, the inspection screen generation unit 28 acquires a snapshot corresponding to the next earlier (late) time stamp. On the other hand, if the determination in step S440 is negative, or if the determination in step S442 is negative, the process proceeds to step S444.

  In step S444, it is determined whether or not the inspection screen generation unit 28 has received a display end request. In this case, when the process of step S526 is performed in the inspection terminal 50, the determination in step S444 is affirmed. If the determination in step S444 is negative, the inspection screen generation unit 28 returns to step S440. However, if the determination in step S444 is affirmative, the inspection screen generation unit 28 performs all the processes in FIGS. Exit.

  As described above, by performing the processes of FIGS. 16 to 18, an inspection screen as shown in FIGS. 19 and 20 is displayed on the inspection terminal 50 according to the position and inspection direction of the field inspector, It will be updated accordingly. Therefore, the field inspector can easily find a portion to be inspected (estimated damage portion) while using the mark image as a mark.

  In the present embodiment, the position information when the photographing apparatus 10 is photographing, the position information of the local inspector, and the position information of the mark image can be acquired using GPS. It is also possible to display a screen as shown in FIG. 21 together with the screens of FIGS. In this case, for example, in the inspection target image position table 64 in FIG. 6B, the shooting position may be managed corresponding to the image snapshot. Thus, by displaying the screen of FIG. 21, the field inspector can more easily identify the estimated damaged portion.

  As described above in detail, according to the present embodiment, the moving image recording unit 22 captures the estimated damaged portion estimated to be damaged on the road surface of the bridge and the vicinity of the estimated damaged portion. The inspection target image specifying unit 24 obtains an instruction that specifies one frame (inspection target image) in which the estimated damaged portion is captured from a plurality of frames included in the video from the consultant terminal 30. The reception and mark image extraction unit 26 extracts a frame (mark image) having a large difference from the inspection target image from frames taken before and after the time when the specified inspection target image is taken, and an inspection screen generation unit 28 generates an inspection screen for simultaneously displaying the mark image and the inspection target image. As a result, in this embodiment, the on-site inspector displays an inspection screen on which a mark image that serves as a mark for searching for an estimated damaged portion with a large difference between the inspection target image and the inspection target image is displayed at the same time. While looking at the site, you can search for the estimated damage area. Thereby, the field inspector can easily specify the position of the estimated damaged portion, and can easily perform the field inspection.

  Further, in the present embodiment, the moving image recording unit 22 acquires shooting information including the direction (gyro value) of the shooting device 10 together with the moving image file from the shooting device 10, and the inspection screen generation unit 28 checks along with the mark image. A screen that displays the shooting direction (left and right) of the mark image with respect to the shooting direction of the target image is generated. Thereby, in this embodiment, the field inspector can specify the position of the estimated damaged portion more easily by referring to the shooting direction of the mark image.

  Further, in the present embodiment, the inspection screen generation unit 28 determines whether or not the road surface is inspected from the same direction as the direction photographed by the photographing apparatus 10 in step S414, and is determined based on the judgment result. An inspection screen is generated based on the display position and display direction of the mark image and the inspection target image. As a result, even if the local inspector inspects the road surface from either the starting point side or the ending point side of the bridge, an inspection screen corresponding to the inspection direction can be provided.

  In the above embodiment, a gyro value may be used when extracting a mark image. Specifically, when comparing the starting image and the preceding and succeeding images within a predetermined value (for example, 2 minutes) in step S310 in FIG. 14, an image in which the gyro value is stable for a predetermined time among the preceding and following images You may make it compare with an inspection object image. By doing so, it is possible to perform the mark image acquisition process in a state where images (images that are highly likely to be blurred) are excluded while the imaging apparatus 10 is changing the imaging direction.

  In addition, although the case where the server 20 was provided was demonstrated in the said embodiment, it is not restricted to this. For example, the consultant terminal 30 or the inspection terminal 50 may have the function of the server 20. Moreover, in the said embodiment, it is good also as using the camera function which the terminal 50 for an inspection has instead of the imaging device 10. FIG.

  In the above-described embodiment, the case where the inspection using the moving image is performed has been described. However, the present invention is not limited to this, and the above-described embodiment can also be applied to the inspection using the still images captured in many places. is there.

  In the above-described embodiment, the case where the structure to be inspected (photographed object) is a road surface of a bridge has been described. However, the present invention is not limited to this. For example, road structures, embankments, river bank slopes, retaining walls for preventing landslides, and walls of buildings such as housing complexes and buildings may be structures to be inspected (photographed).

  The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the processing apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium.

  When the program is distributed, for example, it is sold in the form of a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

  The above-described embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

In addition, the following additional notes are disclosed regarding the above description.
(Additional remark 1) The acquisition process of acquiring the image group which image | photographed with the imaging | photography apparatus the estimated damage part estimated to be damaged among the structures of inspection object, and the vicinity of the said estimated damage part,
An accepting step of accepting an instruction specifying one image in which the estimated damaged portion is photographed among a plurality of images included in the image group;
An extraction step of extracting at least one image having a predetermined difference or more from the image taken before and after the time when the specified image was taken;
A structure inspection support method, wherein a computer executes a generation step of generating a screen for simultaneously displaying the extracted image and the identified image.
(Supplementary Note 2) In the acquisition step, in addition to the image group, information on the orientation of the image capturing device when the image device is imaged is acquired,
The structure according to claim 1, wherein the generation step generates a screen that displays information about a shooting direction of the extracted image with respect to a shooting direction of the specified image together with the extracted image. Inspection support method.
(Supplementary Note 3) Judgment whether to inspect the structure from the same direction as the direction in which the imaging device has taken the image or to inspect the structure from the direction opposite to the direction in which the imaging device has taken an image The computer executes the process;
In the generation step, a display position and a display direction of each of the extracted image and the specified image are determined based on the determination result in the determination step, and the screen is displayed based on the display position and the display direction. The structure inspection support method according to appendix 1 or 2, wherein the structure inspection support method is generated.
(Supplementary Note 4) In the acquisition step, in addition to the image group, information on the position of the photographing device when the image device is photographed is obtained.
In the determination step, the determination is performed using information on the position of the person inspecting the structure and information on the position of the imaging device acquired in the acquisition step. Structure inspection support method.
(Supplementary Note 5) Obtain an image group obtained by photographing an estimated damaged portion estimated to be damaged in a structure to be inspected and a vicinity of the estimated damaged portion with a photographing apparatus,
Receiving an instruction specifying one image in which the estimated damaged portion is photographed among a plurality of images included in the image group;
Extracting at least one image whose difference from the specified image is greater than or equal to a predetermined value from images taken before and after the time when the specified image was taken;
A structure inspection support program for causing a computer to execute processing for generating a screen for simultaneously displaying the extracted image and the identified image.
(Additional remark 6) In the said process to acquire, together with the said image group, the information regarding the direction of the said imaging device when the image group was image | photographed with the said imaging device,
6. The structure according to appendix 5, wherein the generating process generates a screen that displays information about the shooting direction of the extracted image with respect to the shooting direction of the specified image together with the extracted image. Inspection support program.
(Additional remark 7) The process which determines whether the said structure is inspected from the same direction as the direction which the said imaging device image | photographed, or the said structure is inspected from the direction opposite to the direction which the said imaging device image | photographed The computer executes,
In the generation process, the display position and the display direction of each of the extracted image and the specified image are determined based on the determination result in the determination process, and the display position and the display direction are used to determine the display position and the display direction. The structure inspection support program according to appendix 5 or 6, characterized by generating a screen.
(Additional remark 8) In the said process to acquire, together with the said image group, the information regarding the position of the said imaging device when an image group was image | photographed with the said imaging device is acquired,
The appendix 7 is characterized in that in the determination process, the determination is performed using information on a position of the person inspecting the structure and information on the position of the photographing apparatus acquired in the acquisition process. The structure inspection support program described.
(Additional remark 9) The acquisition part which acquires the image group which image | photographed with the imaging | photography apparatus the estimated damage part estimated to be damaged among the structures of inspection object, and the vicinity of the said estimated damage part,
A receiving unit that receives an instruction that identifies one image in which the estimated damaged portion is photographed among a plurality of images included in the image group;
An extraction unit that extracts at least one image having a predetermined difference or more from the image captured before and after the time when the identified image is captured;
A structure inspection support apparatus comprising: a generation unit that generates a screen for simultaneously displaying the extracted image and the identified image.
(Additional remark 10) The said acquisition part acquires the information regarding the direction of the said imaging device when the image group was image | photographed with the said imaging device with the said image group,
The structure according to appendix 9, wherein the generation unit generates a screen that displays information about a shooting direction of the extracted image with respect to a shooting direction of the specified image together with the extracted image. Inspection support device.
(Supplementary Note 11) Judgment to determine whether the structure is inspected from the same direction as the direction in which the imaging device has been imaged or whether the structure is to be inspected from the direction opposite to the direction in which the imaging device has been Part
The generation unit determines a display position and a display direction of each of the extracted image and the specified image based on a determination result of the determination unit, and displays the screen based on the display position and the display direction. The structure inspection support device according to appendix 9 or 10, wherein the structure inspection support device is generated.
(Additional remark 12) The said acquisition part acquires the information regarding the position of the said imaging device when the image group was image | photographed with the said imaging device with the said image group,
The determination unit performs the determination using information on a position of a person inspecting the structure and information on a position of the photographing apparatus acquired by the acquisition unit. Structure inspection support device.

10 imaging device 20 server (structure inspection support device)
22 Movie recording unit (acquisition unit)
24 Inspection target image identification part (reception part)
26 Mark image extraction unit (extraction unit)
28 Screen generator for inspection (generation unit, determination unit)

Claims (5)

An acquisition step of acquiring an image group obtained by photographing an estimated damaged portion estimated to be damaged in a structure to be inspected, and the vicinity of the estimated damaged portion with an imaging device;
An accepting step of accepting an instruction specifying one image in which the estimated damaged portion is photographed among a plurality of images included in the image group;
An extraction step of extracting at least one image having a predetermined difference or more from the image taken before and after the time when the specified image was taken;
A structure inspection support method, wherein a computer executes a generation step of generating a screen for simultaneously displaying the extracted image and the identified image. In the acquisition step, along with the image group, acquire information on the orientation of the image capturing device when the image device is imaged,
2. The structure according to claim 1, wherein in the generation step, a screen is generated that displays information about a shooting direction of the extracted image with respect to a shooting direction of the identified image together with the extracted image. Inspection support method. A determination step of determining whether the structure is inspected from the same direction as the direction photographed by the photographing apparatus or whether the structure is inspected from a direction opposite to the direction photographed by the photographing apparatus; Runs and
In the generation step, a display position and a display direction of each of the extracted image and the specified image are determined based on the determination result in the determination step, and the screen is displayed based on the display position and the display direction. The structure inspection support method according to claim 1, wherein the structure inspection support method is generated. Obtain an image group obtained by photographing the estimated damaged portion estimated to be damaged in the structure to be inspected and the vicinity of the estimated damaged portion with the photographing device,
Receiving an instruction specifying one image in which the estimated damaged portion is photographed among a plurality of images included in the image group;
Extracting at least one image whose difference from the specified image is greater than or equal to a predetermined value from images taken before and after the time when the specified image was taken;
A structure inspection support program for causing a computer to execute processing for generating a screen for simultaneously displaying the extracted image and the identified image. An acquisition unit for acquiring an image group obtained by photographing an estimated damage portion estimated to be damaged among structures to be inspected, and a vicinity of the estimated damage portion, with an imaging device;
A receiving unit that receives an instruction that identifies one image in which the estimated damaged portion is photographed among a plurality of images included in the image group;
An extraction unit that extracts at least one image having a predetermined difference or more from the image captured before and after the time when the identified image is captured;
A structure inspection support apparatus comprising: a generation unit that generates a screen for simultaneously displaying the extracted image and the identified image.

Images