JPWO2018037689A1 - Image processing device - Google Patents

Abstract

  Provided are an image processing apparatus and an image processing method by which a user can easily check a collation between an entire image showing a predetermined range of a structure such as a bridge and an enlarged image of a part of the entire image. An image processing server (10) constituting the image processing apparatus acquires an entire image including one space of the floor slab and an enlarged image obtained by enlarging a defect in the space, and the verification processing unit Then, the entire image and the enlarged image are collated, and a conversion parameter for projective transformation of the enlarged image with respect to the entire image is calculated. The user terminal acquires the conversion parameter from the image processing server (10), and the display control unit of the user terminal includes the enlarged image (150B-1) obtained by projective conversion using the projective conversion formula having the conversion parameter, and the entire image. The candidate image (150B-2) corresponding to the candidate area of the enlarged image is displayed on the display unit so as to be comparable.

Description

  The present invention relates to an image processing apparatus and an image processing method, and more particularly to a technique that makes it easy to compare an entire image of a structure such as a bridge and a partially enlarged image of the entire image.

  Social infrastructure structures such as bridges need to be inspected regularly. In order to efficiently inspect a structure, a crack defect detection method for automatically detecting a defect (crack defect) by processing an image obtained by photographing the structure has been proposed (Patent Document 1). .

  The crack defect detection method described in Patent Document 1 acquires an entire image obtained by orthographic transformation based on the focal length of the camera and the known dimensions of the structure from the entire image obtained by photographing a predetermined range of the structure as a whole. Then, the camera is rotated with the same position as the shooting position of the whole image as the base point, and the partial image obtained by dividing and enlarging the predetermined range of the structure is acquired, and further crack defects within the shooting range of the partial image are identified. An enlarged image obtained by enlarging the image so as to be possible is acquired.

  By collating the orthographically transformed whole image and the partial image, the positional relationship between the whole image and the partial image and the orthographically transformed partial image are obtained, and the orthographically transformed partial image and the enlarged image are collated. Thus, the positional relationship between the partial image and the enlarged image and the enlarged image obtained by orthographic transformation are obtained.

  A crack defect is detected using an enlarged image that has undergone orthographic transformation. Thereby, even under conditions where the structure cannot be photographed from the front, an accurate crack defect can be detected using the enlarged image subjected to orthographic transformation. Further, the positional relationship between the orthographically transformed whole image and the orthographically transformed partial image is obtained, and the positional relationship between the orthographically transformed partial image and the orthographically transformed enlarged image is obtained. Therefore, the position of the crack defect detected from the enlarged image with respect to the entire image (structure) can be specified.

JP 2002-328096 A

  In the crack defect detection method described in Patent Document 1, in order to perform orthogonal projection conversion on the entire image obtained by photographing a predetermined range of the structure, there is a building in which a rectangular part having a known dimension exists within the predetermined range of the building. It is necessary to acquire the dimensions of a rectangular part that is an inspection target and exists within a predetermined range of the building.

  Further, Patent Document 1 discloses a whole image obtained by orthogonal projection transformation, a partial image obtained by rotating the camera with the same position as the photographing position of the whole image as a base point, and dividing and enlarging a predetermined range of the structure. However, there is no description regarding a specific matching method between the whole image and the partial image.

  Similarly, Patent Document 1 has a description for obtaining a mutual positional relationship by collating an orthographic-transformed partial image and an enlarged image, but relates to a specific collation method between the partial image and the enlarged image. There is no description.

  Further, Patent Document 1 does not describe a countermeasure when an error occurs in the collation between the entire image and the partial image or the collation between the partial image and the enlarged image. If an error occurs in the collation between the entire image and the partial image, or the collation between the partial image and the enlarged image, the mutual positional relationship between the images is also incorrect, and as a result, the position where the crack defect is detected (the predetermined range of the structure) There is a problem that the position of the crack defect) is wrong.

  The present invention has been made in view of such circumstances, and allows a user to easily check a collation result between an entire image showing a predetermined range of a structure such as a bridge and a partial enlarged image of the entire image. An object of the present invention is to provide an image processing apparatus and an image processing method.

  In order to achieve the above object, an image processing apparatus according to an aspect of the present invention includes a first image acquisition unit that acquires an entire image indicating a predetermined range of a structure, and enlarges a defect in the structure within the predetermined range. A second image acquisition unit that acquires a magnified image photographed in this manner, a collation processing unit that collates the entire image and the magnified image, and calculates positional relationship information related to a region of the magnified image in the entire image; A display control unit configured to display the enlarged image and a candidate image corresponding to the candidate region of the enlarged image in the entire image specified based on the positional relationship information on the display unit so as to be comparable.

  According to one aspect of the present invention, the entire image and the enlarged image are collated, and the positional relationship information related to the region of the enlarged image in the entire image is calculated. The display control unit causes the display unit to display the enlarged image and the candidate image corresponding to the candidate area of the enlarged image in the entire image specified based on the positional relationship information. The user visually confirms the enlarged image displayed on the display unit so as to be comparable and the candidate image corresponding to the candidate area of the enlarged image in the entire image, thereby confirming whether the comparison result between the entire image and the enlarged image is appropriate. Can be confirmed. Note that, when the entire image and the enlarged image are appropriately checked, the positional relationship information related to the enlarged image region in the entire image can be calculated without error.

  In the image processing device according to another aspect of the present invention, the collation processing unit detects a plurality of corresponding points having the same feature amount from the entire image and the enlarged image, and the enlarged image or the enlarged image based on the detected plurality of corresponding points. It is preferable to calculate conversion parameters for geometrically converting the entire image as positional relationship information.

  In the image processing apparatus according to still another aspect of the present invention, the display control unit combines the enlarged image obtained by geometric conversion based on the conversion parameter with the entire image that is combined with the candidate area corresponding to the enlarged image in the entire image. It is preferable to switch and display the previous whole image on the display unit. The enlarged image geometrically converted based on the conversion parameter has the same shape and the same size as the image of the candidate area corresponding to the enlarged image in the entire image. Then, the entire image obtained by synthesizing the geometrically transformed enlarged image in the candidate area corresponding to the enlarged image in the entire image and the whole image before synthesis are switched and displayed on the display unit, thereby easily comparing the two images. be able to.

  In the image processing apparatus according to still another aspect of the present invention, the display control unit combines the enlarged image obtained by geometric conversion based on the conversion parameter with the entire image that is combined with the candidate area corresponding to the enlarged image in the entire image. It is preferable to display the previous whole image side by side on the display unit. By comparing the entire image obtained by synthesizing the geometrically transformed enlarged image with the candidate area corresponding to the enlarged image in the entire image and the entire image before synthesis on the display unit, both images can be easily compared. .

  In the image processing device according to still another aspect of the present invention, the display control unit includes: an enlarged image; and a converted image obtained by geometrically converting a candidate image corresponding to the enlarged image in the entire image based on the conversion parameter. Is preferably displayed on the display unit. A converted image obtained by geometrically converting a candidate image corresponding to the enlarged image in the entire image based on the conversion parameter has the same shape and the same size as the enlarged image. Then, the enlarged image and the converted image obtained by geometrically transforming the candidate image corresponding to the enlarged image in the entire image are switched and displayed on the display unit, so that both images can be easily compared.

  In the image processing device according to still another aspect of the present invention, the display control unit includes: an enlarged image; and a converted image obtained by geometrically converting a candidate image corresponding to the enlarged image in the entire image based on the conversion parameter. Are preferably displayed side by side on the display unit. By displaying the enlarged image and the converted image obtained by geometrically converting the candidate image corresponding to the enlarged image in the entire image side by side on the display unit, the two images can be easily compared.

  In the image processing apparatus according to still another aspect of the present invention, the image processing apparatus includes a defect information input unit that inputs defect information including the position and shape of the defect of the structure on the enlarged image, and the display control unit is based on the defect information. It is preferable that the defect drawing line obtained by tracing the defect of the structure is superimposed and displayed on the enlarged image and the entire image displayed on the display unit. Thereby, the defect of a structure can be easily visually recognized on an enlarged image and a whole image.

  In the image processing apparatus according to still another aspect of the present invention, the image processing apparatus includes a defect identification unit that performs image analysis of the enlarged image and identifies a defect in the structure, and the defect information input unit inputs defect information from the defect identification unit. Is preferred. Thereby, defect information can be input automatically.

  In the image processing apparatus according to still another aspect of the present invention, it is preferable that the defect information input unit includes a pointing device that manually indicates the position and shape of the defect of the structure on the enlarged image. Thereby, defect information can be input manually.

  In the image processing apparatus according to still another aspect of the present invention, it is preferable to include a correction unit that manually corrects the defect information superimposed and displayed. Thereby, when there is an error in the defect information input from the defect identification unit, or when there is an error in the instruction of the defect information by the pointing device, the defect information can be corrected.

  In the image processing apparatus according to still another aspect of the present invention, a storage unit that stores drawing data of a structure, and a conversion unit that performs geometric conversion that matches a predetermined range of the structure with a corresponding range of the drawing data. Preferably, a conversion unit that geometrically converts only the defect drawing lines superimposed on the entire image and a data output unit that outputs data indicating the defect drawing lines geometrically converted in association with the drawing data are provided. Thereby, drawing data in which a defect drawing line is drawn can be acquired.

  In the image processing apparatus according to still another aspect of the present invention, the defect of the structure is a crack, the defect identifying unit extracts a crack image indicating the crack of the structure from the enlarged image, and the display control unit is cracked. It is preferable to generate a defect drawing line in which an image is emphasized by luminance or color.

  In the image processing apparatus according to still another aspect of the present invention, the defect of the structure is a crack, and the display control unit displays a defect drawing line indicating the crack, a crack width, and a crack indicating an interval between adjacent cracks. Information is preferably displayed on the display unit.

  In the image processing apparatus according to still another aspect of the present invention, the defect of the structure is a crack, and the display control unit can display the defect drawing line indicating the crack in a classified manner according to the width of the crack. preferable. For example, the defect drawing lines are displayed by classification according to the width of cracks (by color coding or selecting crack widths to be displayed).

  In the image processing apparatus according to still another aspect of the present invention, the defect of the structure is a crack, and a determination unit that determines a progress level of the crack based on crack information indicating a width of the crack and an interval between adjacent cracks. Preferably, the display control unit classifies and displays the defect drawing lines indicating the cracks according to the determined progress level of the cracks. The progress level of a structure crack can be determined by the width of the crack and the distance between adjacent cracks. The display control unit classifies (for example, color-codes) and displays defect drawing lines indicating cracks according to the determined progress level of the cracks.

  In the image processing apparatus according to still another aspect of the present invention, it is preferable that the display control unit causes the display unit to display a frame indicating a candidate area corresponding to the enlarged image in the entire image. Thereby, the enlarged image and the candidate image corresponding to the candidate region of the enlarged image in the entire image can be displayed in a more comparable manner.

  In the image processing device according to still another aspect of the present invention, when the second image acquisition unit acquires a plurality of enlarged images, the collation processing unit collates the entire image and the plurality of enlarged images, A plurality of pieces of positional relationship information related to the candidate areas of the plurality of enlarged images are calculated, and the display control unit corresponds to one enlarged image selected from the plurality of enlarged images and the selected one enlarged image. Preferably, the candidate image corresponding to the candidate region of the selected one enlarged image in the entire image specified based on the positional relationship information is displayed on the display unit so as to be comparable. Thereby, even when a plurality of enlarged images are acquired, the enlarged images selected one by one and the candidate images corresponding to the candidate regions of the selected enlarged images in the entire image can be displayed in a comparable manner. .

  In the image processing device according to still another aspect of the present invention, the collation processing unit collates the entire image and the enlarged image, and detects a plurality of candidate areas indicating the enlarged image in the entire image, the plurality of candidate areas are detected. Each of the plurality of related positional relationship information is calculated, and the display control unit includes a plurality of candidate images corresponding to a plurality of candidate areas of the enlarged image and the enlarged image in the entire image specified based on the plurality of positional relationship information. Are preferably displayed on the display unit so as to be comparable, and an image selection unit that accepts selection of one candidate image among a plurality of candidate images corresponding to the plurality of candidate regions displayed on the display unit is preferably provided. When a plurality of candidate areas indicating an enlarged image in the entire image are detected as a result of collating the entire image and the enlarged image, a plurality of candidate images corresponding to a plurality of candidate areas of the enlarged image and the enlarged image in the entire image And can be compared. Then, by accepting the selection of one candidate image among the plurality of candidate images by the image selection unit, the user can select an area corresponding to the enlarged image in the entire image.

  In the image processing apparatus according to still another aspect of the present invention, when the positional relationship information calculated by the matching processing unit is incorrect or when the positional relationship information cannot be calculated, the entire image displayed on the display unit In the above, a position information receiving unit that receives position information of an image corresponding to the enlarged image is provided, and the matching processing unit re-executes matching between the entire image and the enlarged image based on the position information received by the position information receiving unit. It is preferable. The collation processing unit narrows down the collation range in the entire image and re-collates based on the position information of the image corresponding to the enlarged image received by the position information receiving unit (for example, the clicked position information received on the entire image). be able to.

  In the image processing device according to still another aspect of the present invention, the structure is a floor slab of a bridge, the predetermined range is a space between the floor slabs, and the first image acquisition unit is a photograph taken by an imaging device. It is preferable to acquire a floor slab image including two spaces as a whole image. This is because the inspection of the bridge floor slabs is performed at each interval.

  In the image processing device according to still another aspect of the present invention, the first image acquisition unit acquires a plurality of divided images obtained by dividing the space into a plurality of times by the imaging device, and acquires the plurality of divisions acquired. It is preferable to generate a floor slab image by combining images and acquire the generated floor slab image as an entire image. A floor slab image (entire image) including one space needs to be photographed with a resolution that can be collated with at least an enlarged image. In some cases, it is not possible to shoot a floor slab image including a gap. In this case, the space is divided into a plurality of times and photographed to obtain a plurality of divided images. Then, the floor image is generated by synthesizing the acquired plurality of divided images.

  An image processing method according to still another aspect of the present invention includes a first image acquisition step of acquiring an entire image showing a predetermined range of a structure, and an enlarged image obtained by magnifying a defect of the structure within the predetermined range. A second image obtaining step for obtaining an image, a collation processing step for collating the entire image and the magnified image, and calculating positional relationship information related to a region of the magnified image in the entire image, the magnified image, and the positional relationship And a display control step of causing the display unit to display a candidate image corresponding to the candidate region of the enlarged image in the entire image specified based on the information.

  According to the present invention, the entire image showing a predetermined range of a structure such as a bridge is compared with an enlarged image of a part of the entire image, the positional relationship information related to the area of the enlarged image in the entire image is calculated, Since the enlarged image and the candidate image corresponding to the candidate area of the enlarged image in the entire image specified based on the positional relationship information are displayed on the display unit so that the comparison is possible, the user can compare with the display unit. The displayed enlarged image and the candidate image corresponding to the candidate area of the enlarged image in the entire image can be visually confirmed, and the suitability of the matching result between the entire image and the enlarged image can be confirmed.

1 is a conceptual diagram of an image processing apparatus according to the present invention. The block diagram which shows the function structural example of the user terminal 11 It is a figure which shows an example of the image which image | photographed the bridge containing a floor slab, and the figure which shows the several division | segmentation image which image | photographed dividing especially one space of a floor slab in multiple times The figure which shows the whole image 100 containing one space of a floor slab It is a figure which shows an example of the enlarged image which expanded and image | photographed the defect in the predetermined range (space) of a floor slab, and is a figure which shows four enlarged images Block diagram showing a functional configuration example of the image processing server 10 The figure which shows one Embodiment of the display screen containing the some enlarged image displayed on the display part 26 of the user terminal 11 The figure which shows an example of the display screen of the display part 26 for confirming the correspondence of a whole image and an enlarged image. The figure which shows other embodiment of the display screen of the display part 26 for confirming the correspondence of a whole image and an enlarged image. It is a figure which shows an example of the display screen of the display part 26 for confirming the correspondence of an entire image and an enlarged image, and is a figure especially when two candidate images corresponding to an enlarged image exist in the entire image. It is a figure which shows embodiment of the confirmation correction screen after the image analysis of the enlarged image displayed on the display part 26 of the user terminal 11, and is a figure regarding the 1st enlarged image It is a figure which shows embodiment of the confirmation correction screen after the image analysis of the enlarged image displayed on the display part 26 of the user terminal 11, and is a figure regarding the 2nd enlarged image It is a figure which shows embodiment of the confirmation correction screen after the image analysis of the enlarged image displayed on the display part 26 of the user terminal 11, and is a figure regarding the 3rd enlarged image It is a figure which shows embodiment of the confirmation correction screen after the image analysis of the enlarged image displayed on the display part 26 of the user terminal 11, and is a figure regarding the 4th enlarged image The figure used in order to demonstrate the relationship between the whole image 100 including the space | interval image 120 which shows one space, and drawing 200 of a floor slab.

  Embodiments of an image processing apparatus and an image processing method according to the present invention will be described below with reference to the accompanying drawings.

[Image processing device]
FIG. 1 is a conceptual diagram of an image processing apparatus according to the present invention. The image processing apparatus 1 according to the present embodiment includes a user terminal 11 and an image processing server 10 connected to each user terminal 11 via a network 12 such as the Internet.

  The user terminal 11 is a terminal that is operated when the user creates a structure inspection report or the like, and may take the form of a portable terminal such as a personal computer or a tablet device.

  The structure to be inspected in this example is a bridge, and in particular, a case where a floor slab constituting the bridge is to be inspected will be described.

  The bridge is installed in the direction perpendicular to the main girder passed between the abutment and the pier, and the main girder provided in the longitudinal direction of the bridge. It consists of various members including a tilting structure and a horizontal structure that mutually connect the main girders to resist the load, and a floor slab for driving vehicles etc. is placed on the upper part of the main girders etc. Yes. The floor slab is generally made of reinforced concrete.

  The floor slab is usually composed of a rectangular space defined by the main girder and cross girder, and damage to the floor slab (cracking, free lime, peeling of concrete, rebar exposure, etc.) When checking, it is done in units of gaps.

  The image processing server 10 generates information necessary for the user to create an inspection report in accordance with an image taken from the inspection target structure (bridge slab) and instructions sent from the user terminal 11. The generated information is returned to the user terminal 11.

<User terminal 11>
In the image processing apparatus 1, the functional configuration of the user terminal 11 will be described first.

  FIG. 2 is a block diagram illustrating a functional configuration example of the user terminal 11.

  The user terminal 11 of this example mainly includes a terminal communication unit 21, an input / output interface 22, an operation unit 23, a storage unit 24, a display control unit 25, a display unit 26, and a terminal control unit 27.

  The terminal communication unit 21 transmits an image of the floor slab taken via the network 12 (see FIG. 1) and an instruction to execute analysis of the image to the image processing server 10, and the image processing server 10 performs image analysis. A communication unit that receives an analysis result or the like from the image processing server 10 via the network 12.

  The input / output interface 22 is a part that exchanges information with an external device wirelessly or by wire. In this example, an image of a floor slab taken by the digital camera (imaging device) 20 is wirelessly transmitted from the digital camera 20. Input by communication.

  FIG. 3 is a diagram showing an example of an image obtained by photographing a bridge including a floor slab, and particularly shows a plurality of divided images obtained by dividing one space of the floor slab into a plurality of times.

  A plurality (three) of divided images 100A, 100B, and 100C shown in FIG. 3 are images to be combined that are combined into one image by the image processing server 10 as described later. This single image is an image including one space of the floor slab, and hereinafter, “whole image 100 (FIG. 4) showing a predetermined range of the structure (a range corresponding to one space of the floor slab)”. "

  The divided images 100A, 100B, and 100C need to include overlapping regions that overlap between adjacent divided images, and it is preferable that the overlapping regions have about 30% of the total region of each divided image in order to be appropriately combined. . In other words, when the photographer divides and captures one space of the floor slab with the digital camera 20, the angle of view is determined so as to overlap approximately 30% between adjacent divided images. It is preferable to do.

  In FIG. 4, the entire image 100 includes images 102 and 104 showing a part of the main girder, images 112 and 114 showing a part of the horizontal girder, and a part of the floor slab (the main girder and the horizontal girder are displayed. 1 image of a single case) (space image) 120 is included.

  Note that the entire image 100 in this example is an image generated by combining three divided images 100A, 100B, and 100C. For example, the size of the space is small, and the digital camera 20 can perform one shooting. When an image (overall image) of a floor slab including one space can be photographed at an appropriate resolution, an image obtained by one photographing may be used.

  The digital camera 20 may be a general-purpose compact digital camera having a pixel number of about 5000 × 4000 pixels, for example.

  FIG. 5 is a view showing an example of an enlarged image obtained by enlarging a defect in a predetermined range (space) of the floor slab, and in particular, a plurality of images obtained by enlarging a crack which is one of the defects of the floor slab. The enlarged image is shown.

  Multiple (four) enlarged images 150A, 150B, 150C, and 150D shown in FIG. 5 are taken at a resolution that can identify cracks (crack width, length, etc.) that are one of the floor plate defects by image analysis. It has been done. In FIG. 5, reference numerals 30 </ b> A, 30 </ b> B, 30 </ b> C, and 30 </ b> D are crack images showing cracks generated on the floor slab. Note that the crack image 30B shows two cracks, and the crack images 30C and 30D each show a part of one crack.

  For the enlarged image, it is necessary to determine the resolution of the enlarged image according to the minimum width of the crack that needs to be identified by image analysis. For example, when the crack width of 0.1 mm needs to be identified by image analysis, the enlarged image Needs to be photographed at a resolution higher than a crack width of 0.1 mm.

  In addition, when shooting an enlarged image in which a defect is enlarged by the digital camera 20, a method of shooting with a high zoom magnification of the optical zoom of the digital camera 20, a method of shooting near the floor slab, and a combination of these methods There is a way to shoot. In addition, when taking an enlarged image, it is preferable to take a picture facing the floor slab.

  Returning to FIG. 2, the floor slab image (divided image for synthesis shown in FIG. 3, enlarged image shown in FIG. 5) captured by the digital camera 20 is input from the digital camera 20 via the input / output interface 22. The data is taken into the user terminal 11 and stored in the storage unit 24.

  The operation unit 23 includes a mouse, a pointing device including the touch panel when the display unit 26 has a touch panel, and a keyboard, and is used to operate a pointer and an icon displayed on the screen of the display unit 26. A defect information input unit for inputting defect information including the position and shape of the defect of the structure on the enlarged image, a correction unit for manually correcting the defect information, a crack width and a crack indicating an interval between adjacent cracks. It functions as a text information input unit for inputting text information such as information.

  The storage unit 24 stores an image of the floor slab input via the input / output interface 22, and in this example, the user terminal 11 is dedicated for functioning as a part of the image processing apparatus according to the present invention. Software is stored, and further data indicating a defect drawing line to be described later is stored.

  The terminal control unit 27 is a part that executes software stored in the storage unit 24 in response to an input from the operation unit 23 and performs overall control of each unit of the user terminal 11.

  The display control unit 25 controls the entire display screen to be displayed on the display unit 26 based on various data and commands including images input from the terminal control unit 27. The display control unit 25 mainly displays an enlarged image obtained by enlarging a defect of the structure (floor slab) and an enlarged image candidate area in the entire image indicating a predetermined range of the structure (a space between floor slabs). The corresponding candidate image is displayed on the display unit 26 so as to be comparable (display control step). Details of various display screens displayed on the display unit 26 by the display control unit 25 will be described later.

<Image processing server 10>
Next, a functional configuration of the image processing server 10 constituting the image processing apparatus 1 will be described.

  FIG. 6 is a block diagram illustrating a functional configuration example of the image processing server 10.

  The image processing server 10 of this example mainly includes a server communication unit 51, an image DB (DB) 52 that functions as an image storage unit, a drawing DB 53 that stores structure drawing data, a verification processing unit 54, and a defect identification unit. 55, a conversion unit 56, and a server control unit 57.

  The image DB 52 and the drawing DB 53 are configured to include one or a plurality of hard disk devices. The verification processing unit 54, the defect identification unit 55, the conversion unit 56, and the server control unit 57 are stored in one or a plurality of CPUs (Central Processing Units), a CPU working memory, a hard disk device, and the like, and are expanded on the memory. The image processing program is executed, and the CPU executes the image processing program, and functions as each unit.

  The server communication unit 51 receives an image of the floor slab and an instruction to execute analysis of the image from the user terminal 11 via the network 12 (see FIG. 1), an analysis result obtained by image analysis by the image processing server 10, and the like. Is a communication unit that transmits to the user terminal 11 via the network 12.

  The image DB 52 is a storage unit that stores images (divided images, whole images, enlarged images, etc.) received from the user terminal 11 via the server communication unit 51 under the control of the server control unit 57. A plurality of divided images (or entire images) and one or a plurality of enlarged images are stored in association with each other in units of spaces.

  The drawing DB 53 is a storage unit that stores drawing data of structures, and stores and manages drawing data (CAD (computer-aided design) drawings) of a plurality of bridges (including floor slabs). In the drawing DB 53, drawing data transmitted from the user terminal 11 at the time of the initial inspection of the bridge is saved, but the drawing data saving method is not limited to this.

  FIG. 15 shows an entire image 100 including a space image 120 indicating one space, and a floor plan drawing 200 indicated by a dotted line. These whole image 100 and drawing 200 can be displayed on the display unit 26 of the user terminal 11. The floor plan drawing 200 can be displayed on the display unit 26 based on the drawing data stored in the drawing DB 53.

  The floor plan drawing 200 shown in FIG. 15 corresponds to three main girders and five horizontal girders. Mg01 to Mg03 are member numbers of main beams (Mg: Main girder), and Cr01 to Cr05 are respectively member numbers of horizontal beams (Cr: Cross beam). If the entire floor deck drawing of the bridge cannot be displayed properly on one screen (when the total length of the bridge is long), it is possible to display a partial drawing of the floor slab and scroll the floor plan drawing. It is preferable to do so.

  When uploading an image of a floor slab from the user terminal 11 to the image processing server 10, the user transmits information specifying the inspection target bridge (floor slab) to the image processing server 10, and the image processing server 10 Get drawing data. When the image processing server 10 acquires information specifying the floor slab from the user terminal 11, the image processing server 10 can read the corresponding drawing data from the drawing DB 53 and transmit the read drawing data to the user terminal 11.

  The display control unit 25 of the user terminal 11 displays the floor slab drawing 200 (FIG. 15) on the display unit 26 based on the drawing data acquired from the image processing server 10 and also stores the floor slab stored in the storage unit 24. A list of images (a divided image for synthesis, an entire image, and an enlarged image) is displayed.

  The user specifies one space in the floor slab drawing 200, drags and drops a plurality of composite divided images or entire images corresponding to the space, and specifies the space between floor slabs. Are associated with a plurality of divided images or entire images. Similarly, dragging and dropping the enlarged image to the specified space and associating the floor slab space with the enlarged image.

  In the storage unit 24, when the image of the floor slab is managed by a folder for each case, a case in which one case in the drawing 200 is specified and a folder corresponding to the case is specified is specified. It is also possible to associate the floor slab space with the image of each space by dragging and dropping. Further, when uploading a plurality of divided images for synthesis to the image processing server 10, it is preferable to specify and upload the positional relationship within the space between the plurality of divided images 100A, 100B, and 100C as shown in FIG. . This is because the image processing server 10 reduces the calculation load of the synthesis process for synthesizing one whole image 100 from the plurality of divided images 100A, 100B, and 100C.

  As described above, a plurality of divided images or entire images for composition for each space associated with the space between floor slabs are processed from the terminal communication unit 21 of the user terminal 11 via the network 12 and the server communication unit 51. Acquired by the server 10 and stored in the image DB 52 under the control of the server control unit 57. Here, the server communication unit 51, the server control unit 57, and the image DB 52 function as a first image acquisition unit that acquires an entire image (including a plurality of divided images for composition) indicating a floor slab space, This 1st image acquisition part acquires the whole image which shows the space of a floor slab (1st image acquisition process).

  Further, the enlarged image for each space associated with the floor space is acquired from the terminal communication unit 21 of the user terminal 11 to the image processing server 10 via the network 12 and the server communication unit 51, and the server control unit 57. The image is stored in the image DB 52 under the control of the above. Here, the server communication unit 51, the server control unit 57, and the image DB 52 function as a second image acquisition unit that acquires an enlarged image, and the second image acquisition unit detects defects in the floor slab space. An enlarged image obtained by enlarging the image is acquired (second image acquisition step).

  Next, a case where the image processing server 10 acquires a plurality of divided images for synthesis will be described.

  In this case, the server control unit 57 functioning as a part of the first image acquisition unit reads a plurality of divided images for synthesis from the image DB 52, combines the plurality of divided images, and stores one of the floor plates. A floor slab image including a gap is generated, and the generated floor slab image is acquired as an entire image.

  Now, a case will be described in which three divided images 100A, 100B, and 100C shown in FIG. 3 are acquired as a plurality of divided images for synthesis.

  The three divided images 100A, 100B, and 100C are images that are obtained by dividing one space of the floor slab into three portions, and the photographing direction and the photographing position by the digital camera 20 are different, respectively. Between adjacent divided images (the divided image 100A and the divided image 100B, and the divided image 100B and the divided image 100C) include overlapping regions that overlap each other.

  The server control unit 57 detects a plurality of corresponding points having the same feature amount (local feature amount) from overlapping regions between adjacent divided images.

  SIFT (Scale-invariant feature transform) feature, SURF (Speed-Upped Robust Feature) feature, and AKAZE (Accelerated KAZE) feature It has been known.

  It is necessary to detect the number of corresponding points (the number of sets) having the same feature quantity more than the number necessary for calculating the transformation parameter used for geometric transformation of one of the two images. When projective transformation is applied as transformation, four corresponding points are detected.

  The projective transformation formula is as follows.

  The transformation parameters of the projective transformation indicate eight parameters a, b, s, c, d, t, p, and q in the formula [1]. Further, (x, y) and (X, Y) indicate coordinate values before and after projective transformation, respectively.

  Therefore, eight conversion parameters used for projective transformation can be calculated by establishing eight simultaneous equations by substituting the coordinate values of the four corresponding points into the equation [1] and solving the eight simultaneous equations. it can.

  Here, assuming that the divided image 100A is a reference image, a conversion parameter for projective conversion of the divided image 100B when the divided image 100A is used as a reference is calculated, and divided using a projective conversion formula having the calculated conversion parameter. The image 100B is projectively transformed, and the divided image 100A and the divided image 100B obtained by the projective transformation are combined. Similarly, a conversion parameter for projective conversion of the divided image 100C based on the projection-converted divided image 100B is calculated, and the divided image 100C is projectively converted using a projective conversion formula having the calculated conversion parameter. The divided image 100B synthesized by projective transformation and the divided image 100C obtained by projective transformation are synthesized.

  FIG. 4 is a diagram showing the entire image 100 generated by combining the three divided images 100A, 100B, and 100C shown in FIG. 3 as described above. The entire image 100 is stored in the image DB 52, transmitted to the user terminal 11, and stored in the storage unit 24 of the user terminal 11.

  FIG. 7 is a diagram illustrating an embodiment of a display screen including a plurality of enlarged images displayed on the display unit 26 of the user terminal 11.

  On the display screen of the display unit 26 shown in FIG. 7, four enlarged images 150A, 150B, 150C, and 150D that are photographed with enlarged cracks and an “execute analysis” button 40 that is a software button are displayed. .

  When the user requests the image processing server 10 to analyze the enlarged image, the user selects an enlarged image to be analyzed on the screen of the display unit 26 of the user terminal 11 using a pointing device (image selection unit) such as a mouse. Press the “Analysis” button 40. When receiving the enlarged image selection instruction and analysis execution request from the user terminal 11, the image processing server 10 reads the enlarged image selected and instructed from the image group uploaded and registered in the image DB 52, and from the read enlarged image Processing such as detection of various defects (in this example, cracks) is performed.

  Returning to FIG. 6, the collation processing unit 54 collates an image of the floor slab (an entire image) including one space and an enlarged image obtained by enlarging a defect in the space, and enlarges the entire image. This is a part for calculating positional relationship information related to the image area.

  Now, when the enlarged image 150B is selected and an analysis execution request for the enlarged image 150B is received, the collation processing unit 54 collates the entire image 100 (FIG. 4) and the enlarged image 150B (FIG. 7), and enlarges the image 150B. Which part of the entire image 100 is an enlarged image.

  The collation processing unit 54 collates the entire image 100 and the enlarged image 150B by extracting feature amounts (SIFT feature amounts or SURF feature amounts described above) independent of enlargement / reduction and rotation between images. Specifically, the matching processing unit 54 detects a plurality of corresponding points having the same local feature amount between the entire image 100 and the enlarged image 150B, and performs projective transformation on the enlarged image 150B with respect to the entire image 100. Is calculated (refer to the collation processing step, [Expression 1]). The calculated projective transformation formula (conversion parameter) is positional relationship information related to the area of the enlarged image in the entire image.

  In this example, when an analysis execution request for the enlarged image 150B is received, the image processing server 10 uses the conversion parameters of the projective transformation formula calculated by the matching processing unit 54 as a part of the analysis result of the enlarged image 150B. Return to 11.

  The display control unit 25 of the user terminal 11 allows the display unit 26 to compare the enlarged image with a candidate image corresponding to the candidate region of the enlarged image in the entire image specified based on the positional relationship information (conversion parameter). This is the part to be displayed.

  Confirming which part of the whole image (space) the enlarged image that requested the analysis execution is based on the fact that defects (such as cracks) automatically identified from the enlarged image are It is important to identify the position of the throat. Therefore, in the present invention, the user can easily confirm the collation result performed by the collation processing unit 54.

<First Embodiment of Verification Confirmation>
In the first embodiment, the display control unit 25 of the user terminal 11 performs projective transformation on the enlarged image 150B based on the projective transformation formula having the transformation parameters received from the image processing server 10, and performs projective transformation as shown in FIG. The entire image 100-1 obtained by combining the enlarged image 150B-1 with the entire image 100 and the entire image 100 before combining are displayed so as to be switchable (display processing step).

  FIG. 8 is a diagram illustrating an example of a display screen of the display unit 26 for confirming the correspondence between the entire image and the enlarged image.

  The display screen shown in FIG. 8 includes an entire image 100-1 including an enlarged image 150B-1 synthesized by projective transformation, an enlarged image 150B that has requested execution of analysis, a “switch” button 41 that is a software button, and the like. It is displayed.

  Each time the “switch” button 41 is clicked, the display control unit 25 selects the entire image 100-1 including the enlarged image 150B-1 and the original entire image 100 before combining the enlarged image 150B-1 (FIG. 4). Are alternately switched and displayed on the display unit 26.

  In FIG. 8, when displaying the original whole image 100 (see FIG. 4), candidates corresponding to the candidate area of the enlarged image 150B-1 in the original whole image 100, instead of the enlarged image 150B-1 obtained by projective transformation. The image 150B-2 is displayed. In this case, in the entire image 100 shown in FIG. 4, it is unclear which region in the entire image 100 the candidate image 150B-2 is, so a frame indicating a candidate region corresponding to the enlarged image 150B-1 Is preferably superimposed on the entire image 100. Also, when displaying the entire image 100-1 including the enlarged image 150B-1, it is preferable to display a frame indicating the area of the enlarged image 150B-1. This is for clarifying in which region in the entire image 100 the enlarged image 150B-1 is synthesized.

  The user causes the “switch” button 41 to alternately display the entire image 100-1 including the enlarged image 150B-1 (enlarged image 150B-1) and the original entire image 100 (candidate image 150B-2). By comparing the images, it is possible to confirm whether or not the enlarged image 150B-1 and the candidate image 150B-2 are images of the same area of the floor slab.

  Here, since the enlarged image 150B-1 has undergone projective transformation, the enlarged image 150B-1 and the candidate image 150B-2 are displayed with the same size and the same outer shape. Thereby, the user can easily confirm visually whether or not the enlarged image 150B-1 and the candidate image 150B-2 are images of the same area of the floor slab. That is, it can be confirmed whether or not the collation processing unit 54 of the image processing server 10 appropriately collates the entire image and the enlarged image (detects a local feature amount).

  When the enlarged image and the image (candidate image) in the entire image corresponding to the enlarged image are not images of the same area of the floor slab, the search for the enlarged image by the matching processing unit 54 has failed (the candidate image is incorrect )become. That is, the conversion parameter of the projection conversion formula calculated by the matching processing unit 54 is incorrect.

  In this case, the user notifies the image processing server 10 that the search for the enlarged image has failed, and the position of the image corresponding to the enlarged image on the whole image 100 displayed on the display unit 26 is indicated by the pointing device. The designated position is notified to the image processing server 10 as the position information of the image corresponding to the enlarged image. The image processing server 10 includes a position information receiving unit that receives position information of an image corresponding to the enlarged image on the entire image, and the matching processing unit 54 includes position information (for example, the entire image) received by the position information receiving unit. Based on the clicked position information received on the image), the collation range in the entire image is narrowed down and collation is re-executed. Also, when the enlarged image search by the matching processing unit 54 is impossible (when the candidate image is not found), the matching processing unit 54 can re-execute the matching as in the case of the enlarged image search failure.

<Modification of First Embodiment of Collation Confirmation>
In the first embodiment of the verification check, the display control unit 25 combines the entire image 100-1 obtained by combining the enlarged image 150B-1 obtained by projective transformation as shown in FIG. 8 with the entire image 100, and the entire image before combining. 100 is switched and displayed on the display screen of the display unit 26 in accordance with the operation of the “switch” button 41 so that both images can be compared. In the modification of the first embodiment, the display control unit 25 Does not provide the “switch” button 41, and displays the combined whole image 100-1 and the combined whole image 100 side by side on the display screen of the display unit 26.

  As a result, the user compares the entire image 100-1 after combining the enlarged image 150B-1 with the entire image 100 before combining (particularly, the enlarged image 150B-1 and the candidate image 150B-2 in the entire image 100). Confirmation of whether or not the images are of the same floor slab) can be performed more accurately.

<Second Embodiment of Verification Confirmation>
In the second embodiment, the collation processing unit 54 is common in that it detects a plurality of corresponding points having the same local feature amount between the entire image and the enlarged image. The difference is that a projective transformation formula (transformation parameter) for projective transformation of the candidate area corresponding to the enlarged image is calculated.

  Further, the display control unit 25 of the user terminal 11 performs projective transformation on the image of the area corresponding to the enlarged image in the entire image based on the projective transformation formula having the transformation parameter received from the image processing server 10, and is shown in FIG. In this way, the enlarged image 150B and the candidate image 150B-3 obtained by projective transformation of the image corresponding to the enlarged image 150B in the entire image are displayed in a switchable manner (display processing step).

  FIG. 9 is a diagram showing another embodiment of the display screen of the display unit 26 for confirming the correspondence between the entire image and the enlarged image.

  The display screen shown in FIG. 9 includes an enlarged image 150B requested to be analyzed, a candidate image 150B-3 obtained by projective transformation of an image of an area corresponding to the enlarged image 150B in the entire image 100, the entire image 100, and software buttons. A “switch” button 41 and the like are displayed.

  Each time the “switch” button 41 is clicked, the display control unit 25 switches between the enlarged image 150B and the candidate image 150B-3 obtained by projective conversion, and causes the display unit 26 to display the images.

  Since the entire image 100 is also displayed on the display screen illustrated in FIG. 9, the enlarged image 150B or the candidate image 150B-3 obtained by projective transformation and the image corresponding to the enlarged image 150B in the overall image 100 before the projective transformation are displayed. Can be compared. In this case, in the entire image 100, it is preferable that a frame indicating which region in the entire image 100 the candidate image 150B-3 subjected to the projective transformation is superimposed on the entire image 100.

  The user alternately displays an enlarged image 150B requested to be analyzed by the “switch” button 41 and a candidate image 150B-3 obtained by projective transformation of an image of an area corresponding to the enlarged image 150B in the entire image 100. By comparing the images, it is possible to confirm whether or not the enlarged image 150B-1 and the candidate image 150B-3 are images of the same area of the floor slab.

  Here, since the candidate image 150B-3 has undergone projective transformation, the enlarged image 150B and the candidate image 150B-3 are displayed with the same size and the same outer shape. Thereby, the user can easily confirm visually whether or not the enlarged image 150B and the candidate image 150B-3 are images of the same area of the floor slab.

  When the enlarged image and the image (candidate image) in the entire image corresponding to the enlarged image are not images of the same area of the floor slab, the search for the enlarged image by the matching processing unit 54 has failed (the candidate image is incorrect )become.

  In this case, as in the first embodiment of the verification check, the user notifies the image processing server 10 that the search for the enlarged image has failed, and the entire image 100 displayed on the display unit 26. The position of the image corresponding to the enlarged image is designated by the pointing device, the designated position is notified to the image processing server 10 as the position information of the image corresponding to the enlarged image, and the collation processing unit 54 is re-executed. Let it be done.

<Modification of Second Embodiment of Collation Confirmation>
In the second embodiment of the verification check, the display control unit 25 displays the enlarged image 150B and the candidate image 150B-3 obtained by projective transformation as shown in FIG. 9 according to the operation of the “switch” button 41. In the modified example of the second embodiment, the display control unit 25 does not provide the “switch” button 41 and the enlarged image 150B. The candidate images 150B-3 subjected to the projective transformation are displayed side by side on the display screen of the display unit 26.

  As a result, the user can compare the enlarged image 150B with the projective transformed candidate image 150B-3 (particularly, confirm whether the enlarged image 150B and the projective transformed candidate image 150B-3 are images of the same floor slab. ) Can be performed more accurately.

  As described above, when the user designates an enlarged image using the user terminal 11 and requests analysis execution, it is confirmed whether or not the collation in the collation processing unit 54 of the image processing server 10 has been correctly performed. Can do. Further, in the example shown in FIG. 7, since there are four enlarged images 150A, 150B, 150C, and 150D, the enlarged images are selected one by one, the analysis execution is requested, and the above collation confirmation is performed four times. It will be.

<Third embodiment of verification check>
When the matching processing unit 54 detects a plurality of corresponding points whose local feature amounts coincide with the enlarged image 150B in each of a plurality of regions in the whole image 100 (that is, when a plurality of candidate images are detected), the whole image A conversion parameter for projective conversion of the enlarged image 150B with respect to 100 is calculated for each of a plurality of candidate images, and the calculated conversion parameters for each of the plurality of candidate images are returned to the user terminal 11.

  In the third embodiment, the display control unit 25 of the user terminal 11 performs projective conversion on each of the enlarged images 150 </ b> B based on a plurality of projective conversion formulas having conversion parameters for each of a plurality of candidate images received from the image processing server 10. Then, as shown in FIG. 10, the whole image 100-1 obtained by combining the enlarged images 150B-1A and 150B-1B obtained by projective transformation with the whole image 100 and the whole image 100 before being combined are displayed in a switchable manner.

  FIG. 10 is a diagram illustrating an example of a display screen of the display unit 26 for confirming the correspondence between the entire image and the enlarged image. Regarding the case where two candidate images corresponding to the enlarged image exist in the entire image. Show.

  In FIG. 10, when displaying the original whole image 100 (see FIG. 4), the enlarged images 150B-1A, 150B- in the original whole image 100 are used instead of the projection-transformed enlarged images 150B-1A, 150B-1B. Candidate images 150B-2A and 150B-2B corresponding to the candidate area 1B1 are displayed. In this case, in the entire image 100 shown in FIG. 4, it is unclear which region in the entire image 100 is the candidate images 150B-2A and 150B-2B, and therefore the enlarged images 150B-1A and 150B-1B. It is preferable that a frame indicating a candidate area corresponding to is superimposed and displayed on the entire image 100. Further, when displaying the entire image 100-1 including the enlarged images 150B-1A and 150B-1B, it is preferable to display a frame indicating the area of the enlarged images 150B-1A and 150B-1B. This is to clarify in which region in the entire image 100 the enlarged images 150B-1A and 150B-1B are combined.

  The user selects the entire image 100-1 (enlarged images 150B-1A and 150B-1B) including the enlarged images 150B-1A and 150B-1B and the original entire image 100 (candidate images 150B-2A, 150B-2B) are alternately displayed and the two images are compared to determine whether the enlarged image 150B-1A and the candidate image 150B-2A are images of the same area of the floor slab, or the enlarged image 150B- It can be confirmed whether 1B and candidate images 150B-2B are images of the same area of the floor slab.

  The user selects one (correct) candidate image corresponding to the enlarged image from a plurality of candidate images (in this embodiment, two candidate images 150B-2A and 150B-2B), and selects the selection result as an image processing server. 10 is notified.

<Confirmation and correction of defects>
Next, confirmation and correction of a floor slab defect detected by the image processing server 10 will be described.

  In FIG. 6, when the defect identification unit 55 of the image processing server 10 receives an analysis execution request for an enlarged image obtained by enlarging a defect (in this example, a crack) of the floor slab from the user terminal 11, the enlarged image is displayed. By analyzing the image, the crack is identified, and the raster data indicating the identified crack is thinned into vector data. In addition, labeling is performed for each crack, and the vector data of the crack can be managed by the labeling number.

  Then, crack information (defect information) including crack vector data is transmitted from the image processing server 10 to the user terminal 11. Not only the crack vector data, but also raster data indicating a crack image may be transmitted as defect information.

  The display control unit 25 of the user terminal 11 superimposes and displays the defect drawing line obtained by tracing the defect of the floor slab based on the defect information received from the image processing server 10 on the enlarged image and the entire image displayed on the display unit 26.

  FIGS. 11 to 14 are diagrams each showing an embodiment of a confirmation / correction screen after image analysis of an enlarged image displayed on the display unit 26 of the user terminal 11.

  The confirmation / correction screen shown in FIG. 11 functions as the original original enlarged image 150A that requested analysis execution, the enlarged image 150A-4 showing the analysis result, the entire image 100, and the defect information input unit and correction unit. Various icon buttons 42 to 47 and the like are displayed.

  Here, a defect drawing line 30A-1 obtained by tracing a defect (crack) is superimposed and displayed on the enlarged image 150A-4 showing the analysis result. The defect drawing line 30A-1 is preferably a cracked image in which the cracked image 30A of the original enlarged image 150A is distinguishable from the cracked image 30A by brightness or color, or is thickened based on vector data indicating the crack. .

  The user compares the original enlarged image 150A with the enlarged image 150A-4 on which the defect drawing line 30A-1 is displayed in a superimposed manner to determine whether or not the defect (crack) is properly identified by the defect identifying unit 55. Can be confirmed visually.

  Here, when the defect is not automatically recognized, the user selects the icon button 42 indicating the pen, manually operates the cursor (not shown) of the icon button 42, and displays the floor on the enlarged image 150A-4. A defect drawing line (defect information) is added by designating the position and shape of a defect (crack) on the plate. If the automatically recognized cracks are interrupted in the middle, it is also possible to correct these cracks by joining them into a single crack.

  On the other hand, when the floor slab stain or the like is erroneously recognized as a defect (crack), the user selects the icon button 43 indicating the eraser and manually operates the cursor (not shown) of the icon button 43. The defect drawing line (defect information) is erased on the enlarged image 150A-4.

  In addition, not only confirmation and correction of crack defect information but also defect information such as free lime, concrete peeling, and rebar exposure can be confirmed and corrected. In this case, it is preferable to superimpose and display defect drawing lines having different colors and display forms depending on the types of defects.

  The information confirmed and corrected as described above is transmitted to the image processing server 10.

  When the “next” button 44 is clicked on the confirmation / correction screen shown in FIG. 11, a transition is made to the confirmation / correction screen (FIG. 12) for the second enlarged image.

  The second original enlarged image 150B, the enlarged image 150B-4 showing the analysis result, the whole image 100, the icon buttons 42 to 47, and the like are displayed on the confirmation / correction screen shown in FIG.

  The user confirms and corrects the analysis result for the second enlarged image 150B using the confirmation and correction screen shown in FIG. 12 in the same manner as the confirmation and correction of the analysis result for the first enlarged image 150A. Can do. In FIG. 12, a defect drawing line 30B-1 obtained by tracing the automatically recognized crack is superimposed and displayed on the enlarged image 150B-4 showing the analysis result of the original enlarged image 150B.

  After confirming and correcting the analysis result for the second enlarged image 150B on the confirmation / correction screen shown in FIG. 12, when the “next” button 44 is clicked, a confirmation / correction screen for the third enlarged image ( Transition to FIG. When the “return” button 45 is clicked on the confirmation / correction screen shown in FIG. 12, the screen returns to the confirmation / correction screen for the first enlarged image shown in FIG.

  On the confirmation / correction screen shown in FIG. 13, a third original enlarged image 150 </ b> C, an enlarged image 150 </ b> C- 4 showing the analysis result, the entire image 100, icon buttons 42 to 47, and the like are displayed.

  The user confirms and corrects the analysis result for the third enlarged image 150C using the confirmation and correction screen shown in FIG. 13 in the same manner as the confirmation and correction of the analysis result for the first enlarged image 150A. Can do. In FIG. 13, a defect drawing line 30C-1 obtained by tracing the automatically recognized crack is superimposed and displayed on the enlarged image 150C-4 showing the analysis result of the original enlarged image 150C.

  After confirming and correcting the analysis result for the third enlarged image 150C on the confirmation / correction screen shown in FIG. 13, when the “next” button 44 is clicked, a confirmation / correction screen for the fourth enlarged image ( Transition to FIG. When the “return” button 45 is clicked on the confirmation / correction screen shown in FIG. 13, the screen returns to the confirmation / correction screen for the second enlarged image shown in FIG.

  On the confirmation / correction screen shown in FIG. 14, a fourth original enlarged image 150D, an enlarged image 150D-4 showing an analysis result, an entire image 100, icon buttons 42 to 47, and the like are displayed.

  The user confirms and corrects the analysis result for the fourth enlarged image 150D using the confirmation correction screen shown in FIG. 14 in the same manner as the confirmation and correction of the analysis result for the first enlarged image 150A. Can do. In FIG. 14, a defect drawing line 30D-1 obtained by tracing the automatically recognized crack is superimposed and displayed on the enlarged image 150D-4 showing the analysis result of the original enlarged image 150D. Further, in this example, since there are four magnified images obtained by enlarging defects in one space of the floor slab, the analysis result confirmation correction screen for the fourth magnified image 150D shown in FIG. The “Next” button 44 is grayed out so as not to be operated.

  In addition, when the user takes an enlarged image including a floor slab defect, the user measures the width of the crack and the interval between adjacent cracks using a crack scale or the like, and records it in the field book. Then, using the confirmation / correction screen and the operation unit 23 shown in FIGS. 11 to 14, it is possible to input crack information indicating the width of the crack and the interval between adjacent cracks while referring to the field book.

  The display control unit 25 of the user terminal 11 causes the display unit 26 to display the width of the crack and the interval between adjacent cracks based on the crack information input by the operation unit 23. Further, the crack information input at the user terminal 11 is transmitted to the image processing server 10.

<Data output of defect drawing line>
Next, after confirming the positional relationship information in the entire image of the enlarged image and confirming and correcting the defect in the enlarged image, data indicating a defect drawing line to be described in the drawing showing the gap is generated.

  The conversion unit 56 shown in FIG. 6 is a part that performs geometric conversion (projection conversion) on the defect drawing line superimposed on the entire image, and the entire image 100 shown in FIG. 15 is input to the conversion unit 56. The entire image 100 is accompanied by information on defect drawing lines that have been confirmed and corrected.

  When the entire image 100 (or a plurality of divided images) is uploaded from the user terminal 11, the entire image 100 is specified as an image corresponding to any of the floor plan drawings 200. . Further, the rectangular gap image 120 in the entire image 100 can be grasped by performing image analysis on the entire image 100.

  The conversion unit 56 sets four points at the four corners of the rectangular gap image 120 and four points at the four corners corresponding to the drawing 200 as corresponding points, and the gap image 120 corresponds to the drawing 200. A conversion parameter for performing geometric conversion (projective conversion) between the cases is calculated, and the defect drawing line in the case image 120 is projectively converted by a projective conversion formula having the calculated conversion parameter.

  The image processing server 10 transmits data indicating the defect drawing line that has undergone projective transformation by the conversion unit 56 to the user terminal 11.

  The user terminal 11 includes a data output unit (input / output interface 22) that outputs data indicating a defect drawing line that has been projectively transformed so as to correspond to floor plan drawing data in association with the drawing data. By obtaining the drawing data in which the defect drawing lines are combined from the input / output interface 22, it is possible to create an inspection record in which the defect drawing lines are printed on the floor plan drawing.

[Others]
The image processing server 10 includes a determination unit that determines a progress level of cracks in the floor slab based on crack information (a width of cracks and an interval between adjacent cracks) input from the user terminal 11, and the progress level of the cracks is determined by the user terminal. 11, the display control unit 25 of the user terminal 11 preferably classifies and displays the defect drawing lines indicating the cracks according to the progress level of the cracks input from the image processing server 10. Further, the display control unit 25 of the user terminal 11 may classify and display defect drawing lines indicating cracks according to the crack width. For example, a range of crack width (crack width of 2 mm or more) is displayed. By designating, it is possible to selectively display a defect drawing line having a designated crack width.

  In the present embodiment, the image processing apparatus is configured by the image processing server and the user terminal. However, the present invention is not limited thereto, and the image processing apparatus may be realized by an independent apparatus without using a network or the like.

  In the present embodiment, the case of performing projective transformation between the entire image and the enlarged image has been described. However, the present invention is not limited to this, and other geometric transformations such as affine transformation and Helmat transformation can also be used.

  Furthermore, it goes without saying that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 10 Image processing server 11 User terminal 12 Network 20 Digital camera 21 Terminal communication part 22 Input / output interface 23 Operation part 24 Storage part 25 Display control part 26 Display part 27 Terminal control part 30A, 30B, 30C, 30D, 102 104, 112, 114 Image 30A-1, 30B-1, 30C-1, 30D-1 Defect drawing line 40 "Analysis execution" button 41 "Switch" button 42, 43, 46, 47 Icon button 44 "Next" Button 45 “Return” button 51 Server communication unit 52 Image DB
53 Drawing DB
54 collation processing unit 55 defect identification unit 56 conversion unit 57 server control unit 100, 100-1 whole image 100A, 100B, 100C divided image 120 case image 150A, 150A-4, 150B, 150B-1, 150B-1A, 150B -1B, 150B-4, 150C, 150C-4, 150D, 150D-4 Enlarged image 150B-2, 150B-2A, 150B-2B, 150B-3 Candidate image 200 Drawing

Claims (22)

A first image acquisition unit that acquires an entire image indicating a predetermined range of the structure;
A second image acquisition unit for acquiring an enlarged image taken by enlarging a defect of the structure within the predetermined range;
A collation processing unit that collates the entire image and the enlarged image, and calculates positional relationship information related to a region of the enlarged image in the entire image;
A display control unit that causes the display unit to display the enlarged image and a candidate image corresponding to the candidate region of the enlarged image in the entire image specified based on the positional relationship information;
An image processing apparatus.   The collation processing unit detects a plurality of corresponding points having the same amount of feature from the entire image and the enlarged image, and performs a geometric transformation on the enlarged image or the entire image based on the detected plurality of corresponding points. The image processing apparatus according to claim 1, wherein a parameter is calculated as the positional relationship information.   The display control unit combines the whole image obtained by combining the enlarged image geometrically transformed based on the conversion parameter with a candidate area corresponding to the enlarged image in the whole image, and the whole image before the combination. The image processing apparatus according to claim 2, wherein the image processing apparatus is displayed by switching to the display unit.   The display control unit combines the whole image obtained by combining the enlarged image geometrically transformed based on the conversion parameter with a candidate area corresponding to the enlarged image in the whole image, and the whole image before the combination. The image processing apparatus according to claim 2, wherein the image processing apparatus is displayed side by side on the display unit.   The display control unit causes the display unit to switch and display the enlarged image and a converted image obtained by geometrically converting a candidate image corresponding to the enlarged image in the entire image based on the conversion parameter. The image processing apparatus according to claim 2.   The display control unit causes the display unit to display the enlarged image and a converted image obtained by geometrically transforming a candidate image corresponding to the enlarged image in the entire image based on the conversion parameter. Item 3. The image processing apparatus according to Item 2. A defect information input unit for inputting defect information including a position and a shape of a defect of the structure on the enlarged image;
7. The display control unit according to claim 1, wherein a defect drawing line obtained by tracing the defect of the structure based on the defect information is superimposed and displayed on the enlarged image and the entire image displayed on the display unit. The image processing apparatus according to item 1. Image analysis of the enlarged image, comprising a defect identification unit that identifies defects in the structure,
The image processing apparatus according to claim 7, wherein the defect information input unit inputs the defect information from the defect identification unit.   The image processing apparatus according to claim 7, wherein the defect information input unit includes a pointing device that manually indicates a position and shape of a defect of the structure on the enlarged image.   The image processing apparatus according to claim 7, further comprising a correction unit that manually corrects the defect information displayed in a superimposed manner. A storage unit for storing drawing data of the structure;
A conversion unit that performs geometric conversion to match a predetermined range of the structure with a corresponding range of the drawing data, and the conversion unit that performs geometric conversion only on the defect drawing line superimposed and displayed on the entire image;
A data output unit that outputs data indicating the defect drawing line subjected to the geometric transformation in association with the drawing data;
The image processing apparatus according to claim 7, further comprising: The defect of the structure is a crack,
The defect identification unit extracts a crack image indicating a crack of the structure from the enlarged image,
The image processing apparatus according to claim 8, wherein the display control unit generates the defect drawing line in which the cracked image is emphasized by luminance or color. The defect of the structure is a crack,
The display control unit causes the display unit to display the defect drawing line indicating the crack and crack information indicating a width of the crack and an interval between adjacent cracks. The image processing apparatus described. The defect of the structure is a crack,
The image processing apparatus according to claim 7, wherein the display control unit classifies and displays the defect drawing lines indicating the cracks according to a width of the cracks. The defect of the structure is a crack,
A discriminating unit for discriminating a progress level of the crack based on crack information indicating a width of the crack and an interval between adjacent cracks;
The image processing apparatus according to claim 7, wherein the display control unit classifies and displays the defect drawing line indicating the crack according to the determined progress level of the crack.   The image processing apparatus according to claim 1, wherein the display control unit causes the display unit to display a frame indicating a candidate area corresponding to the enlarged image in the entire image. When the second image acquisition unit acquires a plurality of the magnified images, the collation processing unit collates the whole image with the plurality of magnified images, and the candidate areas of the plurality of magnified images in the whole image Multiple pieces of positional relationship information related to
The display control unit includes the one enlarged image selected from the plurality of enlarged images, and the positional relationship information corresponding to the selected one enlarged image. 17. The image processing apparatus according to claim 1, wherein a candidate image corresponding to a candidate area of the selected one enlarged image is displayed on the display unit so as to be comparable. The collation processing unit collates the entire image and the enlarged image, and when detecting a plurality of candidate areas indicating the enlarged image in the entire image, a plurality of pieces of positional relationship information related to the plurality of candidate areas are obtained. Respectively,
The display control unit can compare the enlarged image with a plurality of candidate images corresponding to a plurality of candidate areas of the enlarged image in the whole image specified based on the plurality of positional relationship information. Displayed in the
The image according to any one of claims 1 to 17, further comprising an image selection unit that receives selection of one candidate image among a plurality of candidate images corresponding to the plurality of candidate regions displayed on the display unit. Processing equipment. When the positional relationship information calculated by the matching processing unit is incorrect or when the positional relationship information cannot be calculated, it corresponds to the enlarged image on the whole image displayed on the display unit. A position information receiving unit for receiving image position information;
The image processing apparatus according to any one of claims 1 to 18, wherein the collation processing unit re-executes collation between the entire image and the enlarged image based on the position information received by the position information receiving unit. The structure is a floor slab of a bridge, and the predetermined range is a space between the floor slabs,
The image processing device according to any one of claims 1 to 19, wherein the first image acquisition unit acquires a floor slab image including the one space photographed by an imaging device as the entire image.   The first image acquisition unit acquires a plurality of divided images taken by dividing the space into a plurality of times by the imaging device, and combines the acquired plurality of divided images to obtain the floor slab image. The image processing apparatus according to claim 20, wherein the image processing apparatus generates and acquires the generated floor slab image as the entire image. A first image acquisition step of acquiring an entire image showing a predetermined range of the structure;
A second image acquisition step of acquiring an enlarged image photographed by enlarging a defect of the structure within the predetermined range;
A collation processing step of collating the entire image and the enlarged image, and calculating positional relationship information related to a region of the enlarged image in the entire image;
A display control step of causing the display unit to display the enlarged image and a candidate image corresponding to the candidate area of the enlarged image in the entire image specified based on the positional relationship information;
An image processing method including:

Images