JP2018124832A - Image processing device, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device, an image processing method, and a program that can quickly and accurately perform an operation of grouping for each inspection target in a group of photographed images where a plurality of photographing objects are photographed and a plurality of photographed images exist for each photographing object.SOLUTION: An image processing device 200 groups a photographed image group including photographed images of a plurality of objects to be photographed into groups for each object to be photographed. The image processing device 200 includes an image input unit 201 that the photographed image group is input, a parent image receiving unit 203 that receives designation of a parent image representing a whole image of the object to be photographed, a child image specifying unit 205 that specifies a child image of the object obtained by photographing a part of the object of the parent image to be photographed, and a group generation unit 207 that generates a group composed of a parent image and a child image that have photographed a same object to be photographed or a group composed of the child images for each of the objects to be photographed.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing device, an image processing method, and a program, and more particularly to a technique for classifying a captured image group including captured images of a plurality of shooting targets for each shooting target.

  2. Description of the Related Art Conventionally, there has been proposed a technique related to grouping for searching for a captured image or grouping captured images in a captured image group composed of a plurality of captured images.

  For example, in Patent Document 1, in addition to physical components of color and shape, a symbol related to whether or not a human is visually similar is assigned to an image stored in a database, and the image Techniques used for searching have been proposed.

  Further, for example, in the technique of creating a photo book and a photo album, for each captured image group including captured images captured at a plurality of events input by the user, for each event based on information regarding the captured time of each captured image. A technique for grouping images is applied.

JP 2004-334594 A

  In recent years, photographed images are often used for building inspection work. When a photographed image is used in the inspection work, photographing is performed a plurality of times by changing photographing locations and photographing conditions for each inspection target. That is, a captured image is acquired by changing the shooting location and shooting conditions for each of the plurality of inspection targets. As described above, when a plurality of photographed images are acquired for each inspection object among the plurality of inspection objects, a photographed image group composed of an enormous number of photographed images is generated.

  As a specific example, when checking the floor slab of a bridge, the floor slab is photographed for each gap, but the entire image of the gap is photographed, and the damaged part of the gap is zoomed in and photographed. Then, a plurality of photographed images are acquired for one space (photographing target). In addition, since the bridge is inspected with respect to a plurality of spaces, the inspector (user) obtains a photographed image group composed of a large number of photographed images of a plurality of photographing objects. In order to use the photographed image for inspection, the inspector needs to group the obtained photographed image group for each floor slab (photographing target) to be inspected.

  The grouping operation for each photographing target has been conventionally performed manually by an inspector visually.

  However, when the inspector manually performs grouping, it takes time for the grouping, and when there are a large number of inspection targets, the grouping operation may take a huge amount of time. In addition, in the manual grouping by the inspector, a correct grouping result may not be obtained due to human error.

  Here, each of the captured images constituting the captured image group acquired in the case of inspection is unlikely to have a significantly different shooting time for each event as in the above-described photo book and photo album, and the captured images simply have. In many cases, it is difficult to group by inspection object only by the difference in time.

  Further, Patent Document 1 does not mention that a group of images stored in a database is quickly and accurately grouped for each photographing target.

  The present invention has been made in view of such circumstances, and an object of the present invention is to group each inspection object in a photographed image group in which a plurality of photographed objects are photographed and a plurality of photographed images exist for each photographed object. It is an object to provide an image processing apparatus, an image processing method, and a program that can perform a work to be performed quickly and accurately.

  In order to achieve the above object, an image processing apparatus according to one aspect of the present invention is an image processing apparatus that groups a group of captured images including captured images of a plurality of shooting targets into groups for each shooting target, An image input for inputting a photographed image group including one or a plurality of first images first photographed for each photographing object and a plurality of second images photographed second under photographing conditions different from the first photographing. Based on the received image characteristics of the parent image and the image characteristics of the second image based on the received image characteristics of the parent image and the parent image receiving section. A child image specifying unit that specifies a second image obtained by shooting a part of the shooting target as a child image, and a group composed of a parent image and a child image obtained by shooting the same shooting target for each of the shooting targets; or Consists of child images And a group generator to generate a group.

  According to this aspect, the captured image group includes one or a plurality of first images that are first captured for each of the capturing targets, and a plurality of second images that are captured second under a capturing condition different from the first capturing. Can be grouped according to the parent-child relationship of image features for each subject. That is, the parent image accepting unit accepts designation of a parent image representing the entire image to be photographed, and the child image identifying unit photographs a part of the same photographing object based on the image characteristics of the designated parent image. A child image is identified, and a group composed of a parent image and a child image obtained by photographing the same photographing object or a child image obtained by photographing the same photographing object is generated. Thus, according to the present aspect, the captured image group is grouped for each imaging target based on the image feature of the parent image and the image feature of the child image, so that the grouping of the captured image group can be performed more quickly and accurately. .

  Preferably, the parent image receiving unit receives a plurality of first images obtained by dividing and shooting the shooting target as a parent image, and the child image specifying unit performs a synthesis process on the plurality of first images received as the parent image to obtain a shooting target. A single image representing the whole image is synthesized and a child image is specified based on the image characteristics of the single image and the image characteristics of the second image.

  According to this aspect, the parent image receiving unit receives a plurality of first images obtained by dividing and shooting the shooting target as parent images, and the child image specifying unit displays the received plurality of parent images as a whole image of the shooting target. A child image is identified based on the image characteristics of the single image that is combined with the single image that is represented. As a result, even if the subject to be photographed is divided and photographed largely, the child images are identified based on the image features of the images obtained by compositing the separately photographed images. Accurate grouping can be performed.

  Preferably, the child image specifying unit extracts a feature point between the second image and the parent image, and specifies the child image based on the correspondence relationship between the extracted feature points.

  According to this aspect, the feature point between the second image and the parent image is extracted by the child image specifying unit, and the child image is specified based on the correspondence relationship between the extracted feature points. In addition, grouping of photographed image groups can be performed.

  Preferably, each of the plurality of photographed images constituting the photographed image group has information regarding the photographing time, and the child image specifying unit is photographed after or before the time when the parent image of one group is photographed. The processed second image is set as a child candidate image of the child image, and the child image is specified from the child candidate image.

  According to this aspect, in the case where each of the captured images constituting the captured image group has information regarding the time at which the captured images are captured, the child image specifying unit performs the time after or before the time at which the parent image of one group is captured. The taken second image is used as a child candidate image of the child image, and the child image is specified from the child candidate image. As a result, in this aspect, the child image is specified from the child candidate images by the child image specifying unit, so that the group of captured image groups can be accurately and quickly performed.

  Preferably, each of the plurality of photographed images constituting the photographed image group has information related to the photographed time, and the child image specifying unit includes the time when the parent image of the first group is photographed and the second group of images. The second image taken between the time when the parent image was taken is taken as the child candidate image of the first group of child images, and the child image is specified from the child candidate image.

  According to this aspect, when each of the captured images constituting the captured image group has information regarding the time at which the captured image was captured, the time at which the parent image of the first group was captured by the child image specifying unit, and the second group The second image taken between the time when the parent image is taken is set as a child candidate image of the child image of the first group, and the child image is specified from the child candidate image. As a result, in this aspect, the child image is specified from the child candidate images by the child image specifying unit, so that the group of captured image groups can be accurately and quickly performed.

  Preferably, each of the plurality of photographed images constituting the photographed image group has information regarding the photographing condition, and the parent image receiving unit extracts and extracts the first image from the photographed image group based on the photographing condition. A parent image is received from the first images.

  According to this aspect, when each of the plurality of captured images constituting the captured image group has information regarding the capturing condition, the parent image receiving unit extracts the first image from the captured image group based on the capturing condition. The parent image is received from the extracted first image. Thereby, since this aspect extracts the 1st image which is a candidate selected as a parent image, and selection of a parent image is received from the extracted 1st image, a quick and accurate picked-up image is received. Group grouping can be performed.

  Preferably, the photographed image group includes a landmark image indicating that the photographing target of the photographed image is changed to another photographing target, and each of the plurality of photographed images and the landmark image constituting the photographed image group relates to a time when the photographed image is photographed. The parent image receiving unit receives the first image taken immediately after or immediately before the landmark image is taken as a parent image.

  According to this aspect, the captured image group includes a landmark image indicating that the captured object of the captured image is changed to another captured object, and each of the plurality of captured images and the landmark image constituting the captured image group is captured. When there is information regarding the time, the parent image accepting unit accepts the first image photographed immediately after or just before the landmark image is photographed as the parent image. Thus, according to the present aspect, the first image immediately before or immediately after the landmark image is automatically accepted as the parent image, so that it is possible to quickly and accurately group the captured image group.

  Preferably, the image processing apparatus includes a display control unit that causes the display unit to display a parent image and a child image for each group generated by the group generation unit, and a correction instruction reception unit that receives a correction instruction for the child image. The group generation unit generates the group again based on the correction instruction.

  According to this aspect, the display control unit displays the parent image and the child image on the display unit for each group generated by the group generation unit, the correction instruction receiving unit receives the child image correction instruction, and the group generation unit Thus, the group is generated again based on the correction instruction. Thereby, this aspect can perform accurate grouping of the photographed image group.

  Preferably, the image processing apparatus includes a compositing target specifying unit that specifies a plurality of child images constituting the group as image compositing target images.

  Preferably, the image processing apparatus includes a child image synthesis unit that performs image synthesis on the target image for image synthesis specified by the synthesis target specifying unit.

  According to this aspect, since the child image composition unit composites the target images for image composition, the grouping result is effectively used.

  An image processing method according to another aspect of the present invention is an image processing method for grouping photographed image groups including photographed images of a plurality of photographing objects into groups for each photographing object, wherein the first photographing is performed for each of the photographing objects. An image input step in which a captured image group including a single image or a plurality of first images and a plurality of second images captured under a photographing condition different from the first photographing is input; Based on the received parent image receiving step and the received image characteristics of the parent image and the image characteristics of the second image, a part of the shooting target of the parent image is captured. A child image specifying step for specifying the second image as a child image, and for each of the shooting targets, a group consisting of a parent image and a child image obtained by shooting the same shooting target, or a group consisting of child images Including a group generating step of generating.

  A program that causes a computer to execute an image processing method according to another aspect of the present invention is a program that causes a computer to execute an image processing method that groups a group of captured images including captured images of a plurality of shooting targets into groups for each shooting target. A captured image group including one or a plurality of first images captured for each of the capturing targets and a plurality of second images captured under a capturing condition different from the first capturing is input. Based on the received image feature, the parent image accepting step for accepting any one of the first images as a parent image representing the whole image to be photographed, and the image features of the accepted parent image and the second image. A child image specifying step of specifying a second image obtained by shooting a part of the parent image shooting target as a child image, and shooting the same shooting target for each of the shooting targets Including a group generating step of generating a group consisting group or a child image, and a parent image and child images, a.

  According to the present invention, the parent image accepting unit accepts designation of a parent image representing the entire image to be photographed, and the child image specifying unit accepts the same photographing object based on the image characteristics of the designated parent image. A group of photographed images is grouped because a child image obtained by capturing a part is identified and a group consisting of a parent image and a child image obtained by photographing the same object or a child image obtained by photographing the same object is generated. Can be performed more quickly and accurately.

It is the perspective view which shows the structure of a bridge, and is the perspective view which looked at the bridge from the bottom. It is a perspective view which shows the external appearance structure of a computer. It is a perspective view which shows the external appearance structure of a computer. It is a block diagram which shows the system configuration | structure of a computer. It is a block diagram which shows the function with which an image processing apparatus is provided. It is a figure which shows notionally the picked-up image which image | photographed the space between wide angles. It is a figure explaining the wide-angle image at the time of carrying out the division | segmentation imaging | photography of the space which is imaging | photography object. It is a figure which shows an example of a mark image notionally. It is a figure which shows notionally the telephoto image which telephotographed the space. It is a figure which shows a picked-up image group. It is a figure shown regarding a grouping result. It is a figure shown regarding a grouping result. It is a figure shown regarding a grouping result. It is a figure shown regarding a grouping result. It is a flowchart which shows operation | movement of an image processing apparatus. It is a block diagram which shows the function of an image processing apparatus. It is a figure explaining correction of a child image. It is a figure explaining correction of a child image. It is a block diagram which shows the function of an image processing apparatus. It is a figure explaining the image composition of a plurality of grouped child images.

  Hereinafter, preferred embodiments of an image processing apparatus, an image processing method, and a program according to the present invention will be described with reference to the accompanying drawings.

  In the present invention, it is possible to group a photographic image group composed of photographic images obtained by photographing a plurality of photographic subjects for each photographic subject. Here, the object to be imaged is not particularly limited, but examples include an object to be imaged when a periodic inspection of a structure or a building is performed using a captured image. In the following description, when the bridge 1 (FIG. 1) is inspected as a specific example, a captured image group U (FIG. 10) configured by captured images obtained by capturing a plurality of spaces (a floor slab 6). The grouping will be described.

  FIG. 1 is a perspective view showing a structure of a bridge that is one of structures to be imaged (inspected), and is a perspective view of the bridge as viewed from below.

  A bridge 1 shown in FIG. 1 has a main girder 2, a cross girder 3, a counter tilting structure 4, and a horizontal structure 5, which are connected by bolts, rivets or welding. In addition, a floor slab 6 for driving a vehicle or the like is provided on the upper portion of the main beam 2 or the like. The floor slab 6 is generally made of reinforced concrete.

  The main girder 2 is a member that is passed between the abutment or the pier and supports the load of the vehicle or the like on the floor slab 6. The cross beam 3 is a member that connects the main beams 2 in order to support the load by the plurality of main beams 2. The anti-tilt frame 4 and the horizontal frame 5 are members for connecting the main girders 2 to each other in order to resist lateral loads of wind and earthquake, respectively.

  When the bridge 1 is inspected, for example, a photographed image is acquired for each space to inspect whether there is a damaged portion such as a crack. Here, the space is a space defined by the two main girders 2 facing each other and the two cross beams 3 facing each other.

  Next, the computer 300 provided with the image processing apparatus of the present invention will be described.

  2 and 3 are perspective views showing an external configuration of the computer 300, FIG. 2 is a front perspective view, and FIG. 3 is a rear perspective view.

  As shown in FIGS. 2 and 3, the computer 300 is configured by a tablet computer. The computer 300 includes a flat housing 301 having a rectangular outline, and the housing 301 includes a touch panel display 302 that also serves as a display unit 326 and an input unit 328, operation buttons 303, a speaker 304, a built-in camera 305, an external device. A connection terminal 306 and the like are provided.

  FIG. 4 is a block diagram illustrating a system configuration of the computer 300.

  As shown in FIG. 4, the computer 300 includes a CPU (Central Processing Unit) 310 that controls the overall operation of the computer 300. A main memory 314, a nonvolatile memory 316, a mobile device is connected to the CPU 310 via a system bus 312. A communication unit 318, a wireless LAN communication unit 320, a short-range wireless communication unit 322, a wired communication unit 324, a display unit 326, an input unit 328, a key input unit 330, an audio processing unit 332, an image processing unit 334, and the like are connected. Is done.

  The CPU 310 reads out an operation program (OS (Operating System) and application program that operates on the OS), standard data, and the like stored in the non-volatile memory 316, expands them in the main memory 314, and performs this operation. By executing the program, it functions as a control unit that controls the operation of the entire computer.

  The main memory 314 is constituted by a RAM (Random Access Memory), for example, and functions as a work memory for the CPU 310.

  The nonvolatile memory 316 is configured by, for example, a flash EEPROM (EEPROM: Electrically Erasable Programmable Read Only Memory), and stores the above-described operation program and various fixed data. The nonvolatile memory 316 functions as a storage unit of the computer 300 and stores various data.

  The mobile communication unit 318 is a third generation mobile communication system compliant with the IMT-2000 standard (International Mobile Telecommunication-2000) and a fourth generation mobile communication system compliant with the IMT-Advanced standard (International Mobile Telecommunications-Advanced). Based on this, data is transmitted to and received from the nearest base station (not shown) via the antenna 318A.

  The wireless LAN communication unit 320 uses a predetermined wireless LAN communication standard (for example, IEEE802.11a / b / g / n standard) between the wireless LAN access point and an external device capable of wireless LAN communication via the antenna 320A. ) To perform wireless LAN communication.

  The short-range wireless communication unit 322 transmits / receives data to / from another Bluetooth (registered trademark) device within the range of, for example, class 2 (within a radius of about 10 m) via the antenna 322A.

  The wired communication unit 324 performs communication according to a predetermined communication standard with an external device connected via a cable via the external connection terminal 306. For example, USB communication is performed.

  The display unit 326 includes a color LCD (liquid crystal display) panel that constitutes a display part of the touch panel display 302 and a driving circuit thereof, and displays various images.

  The input unit 328 constitutes a touch panel portion of the touch panel display 302. The input unit 328 is configured integrally with the color LCD panel using a transparent electrode. The input unit 328 generates and outputs two-dimensional position coordinate information corresponding to the inspector's touch operation.

  The key input unit 330 includes a plurality of operation buttons 303 provided on the casing 301 of the computer 300 and a drive circuit thereof.

  The audio processing unit 332 converts the digital audio data provided via the system bus 312 into an analog signal and outputs it from the speaker 304.

  The image processing unit 334 digitizes an analog image signal output from the built-in camera 305 provided with a photographing lens and an image sensor, performs necessary signal processing, and outputs it.

<First Embodiment>
Next, functions provided in the image processing apparatus 200 will be described.

  FIG. 5 is a block diagram illustrating functions provided in the image processing apparatus 200. The image processing apparatus 200 is provided in the computer 300, and each function of the image processing apparatus 200 is realized by the computer 300.

  The image processing apparatus 200 includes an image input unit 201, a parent image reception unit 203, a child image specification unit 205, and a group generation unit 207.

  The image input unit 201 receives a captured image group U that is grouped for each subject. Here, the photographed image group U includes a plurality of photographed images for each photographing object with respect to a plurality of photographing objects. Specifically, the image input unit 201 includes one or a plurality of wide-angle images (first images) 100 (see FIG. 6) that have been subjected to wide-angle imaging (first imaging), and telephoto imaging (second imaging). A photographed image group U including a plurality of photographed telephoto images (second images) 110 (see FIG. 9) is input. Here, the wide-angle shooting and the telephoto shooting have different focal lengths in the shooting conditions. The wide-angle image 100 is wide-angle shot under a short focal length, and the telephoto image 110 is telephoto shot under a long focal length. Has been done.

  The parent image receiving unit 203 receives one of the wide-angle images 100 as a parent image representing the entire image to be captured. That is, the parent image receiving unit 203 receives a designation as a parent image for the wide-angle image 100 in which the entire image to be photographed is captured from the inspector. For example, the parent image receiving unit 203 receives a designation as a parent image for the wide-angle image 100 when the photographed image group U or the wide-angle image 100 displayed on the display unit 326 is selected by the inspector.

  Here, the wide-angle image 100 will be described. FIG. 6 is a diagram conceptually showing an image (wide-angle image 100) obtained by photographing the space at a wide angle.

  When the bridge 1 is inspected, the floor slab 6 is often inspected at intervals. In this case, a wide-angle image 100 obtained by capturing the entire image of the interlace by wide-angle shooting (first shooting) is acquired. In the wide-angle image 100, the floor slab 6 in a range (a space) surrounded by the horizontal beam 3 and the main beam 2 is an object to be imaged, and the entire image of the space is captured. When actually inspecting the bridge 1, wide-angle images 100 are acquired for a plurality of spaces, respectively.

  Further, the parent image receiving unit 203 may receive a plurality of wide-angle images 100 obtained by dividing and shooting the shooting target as parent images. In other words, even when the subject is large and wide-angle photography is performed, if the whole image of the subject cannot be accommodated in one photographed image, the divided images are taken. The wide-angle image 100 that has been divided and photographed may be accepted as a parent image. In this case, the child image specifying unit 205 combines the plurality of wide-angle images 100 received as the parent image into a single image representing the entire image to be photographed, and the image features of the single image. A child image is specified based on the image characteristics of the telephoto image 110.

  FIG. 7 is a diagram for explaining the wide-angle image 100 when the space to be imaged is divided and imaged.

  When the space H to be imaged is large, divided shooting is performed, and a divided wide-angle image 101A, a divided wide-angle image 101B, and a divided wide-angle image 101C are acquired. The parent image receiving unit 203 receives designation of the divided wide-angle image 101A, the divided wide-angle image 101B, and the divided wide-angle image 101C as the parent image. In this case, the child image specifying unit 205 combines the divided wide-angle image 101A, the divided wide-angle image 101B, and the divided wide-angle image 101C, and then specifies the child image based on the image characteristics of the combined image (parent image).

  Further, when each of the plurality of captured images constituting the captured image group U input to the image input unit 201 has information regarding the shooting conditions, the parent image receiving unit 203 is based on the shooting conditions. The wide-angle image 100 may be extracted from the image, and the parent image may be received from the extracted wide-angle image 100. That is, each of the photographed images constituting the photographed image group U has information related to the photographing condition (focal length), and the wide-angle image 100 is extracted from the photographing condition, and the extracted wide-angle image 100 is displayed on the display unit 326. Then, the inspector may select the parent image, and the parent image receiving unit 203 may receive the designation of the parent image.

  In addition, the parent image receiving unit 203 includes a landmark image 112 indicating that the subject to be photographed is changed in the photographed image group U input to the image input unit 201 and includes a plurality of photographed images and landmarks constituting the photographed image group U. When each of the images 112 has information regarding the time at which the image 112 was captured, the parent image receiving unit 203 may receive a wide-angle image captured immediately after or just before the landmark image 112 is captured as the parent image. That is, when the captured image group U includes the landmark image 112, the parent image accepting unit 203 may automatically or manually accept the wide-angle image 100 immediately after or just before the landmark image 112 as the parent image. Specifically, in the captured image group U shown in FIG. 10, the wide-angle image 100 immediately after the landmark image 112 is designated as the parent image.

  Here, the landmark image 112 will be described. FIG. 8 is a diagram conceptually illustrating an example of the landmark image 112 included in the captured image group U. The landmark image 112 is an image photographed to show that the photographing object is switched. That is, when the bridge 1 is inspected, a large number of gaps are photographed, but the landmark image 112 can be used to identify which gap is photographed before or after. In the landmark image 112 shown in FIG. 8, the plate with the number of the case is photographed.

  Other examples of the mark image 112 include an image obtained by photographing a plate on which the name or member number of the bridge 1 is written, and a black image obtained by photographing with the lens cover. The mark image 112 may be included in the captured image group U or may not be included in the captured image group U.

  Returning to FIG. 5, the child image specifying unit 205 uses, as a child image, the telephoto image 110 obtained by capturing a part of the subject of the parent image based on the received image feature of the parent image and the image feature of the telephoto image 110. Identify. That is, the child image specifying unit 205 calculates the correlation (degree of coincidence) between the image feature of the parent image and the image feature of the telephoto image 110 that have been designated by the inspector, and specifies the child image. Specifically, the child image specifying unit 205 extracts feature points between the telephoto image 110 and the parent image, and the telephoto image has a degree of matching between the feature points of the parent image and the telephoto image 110 higher than a predetermined value (threshold value). The image 110 is specified as a child image. A known image processing technique is applied to the extraction of the image features of the parent image and the feature points of the telephoto image 110 and the calculation of the correlation between the image features of the parent image and the feature points of the telephoto image 110 performed by the child image specifying unit 205. The

  In addition, SIFT (Scale-invariant feature transform) feature, SURF (Speed-Upped Robust Feature) feature, and AKAZE (Accelerated KAZE) feature are robust local features that are strong against scaling, rotation, and illumination changes between images. The amount is known.

  Here, the telephoto image 110 will be described. FIG. 9 is a diagram conceptually showing a telephoto image 110 obtained by telephoto shooting the space. When performing inspections, if there are cracks that cannot be detected just by taking a wide-angle image of the gap, or if you want to analyze cracks in more detail, perform telephoto shooting to obtain an enlarged image of the subject to be shot. . Moreover, the telephoto image 110 may be acquired over the whole area of a case, or only a location with a case may be acquired.

  In addition, when the child image specifying unit 205 has information regarding the time when each of the plurality of captured images constituting the captured image group U is captured, the child image specifying unit 205 captures the parent image of one group. The telephoto image 110 taken after or before the time may be used as a child candidate image of the child image, and the child image may be specified from the child candidate image. Specifically, in the photographed image group U shown in FIG. 10, a child image is specified by using a telephoto image photographed after or before the time when the parent image represented by P1 or P2 is photographed as a child candidate image. May be. Thereby, the child image specifying unit 205 does not need to extract the feature points of all the captured images of the captured image group U or all the telephoto images 110 and check the correspondence relationship with the parent image, and the parent image is captured. The telephoto image 110 taken after or before the time may be used as a child candidate image of the child image, and feature points may be extracted for only the child candidate image to check the correspondence.

  Further, when the child image specifying unit 205 has information regarding the time at which each of the plurality of captured images constituting the captured image group U is captured, the child image specifying unit 205 captures the first group of parent images. A telephoto image 110 taken between the time when the second group parent image was taken as a child candidate image of the first group child image, and identifying the child image from the child candidate image It may be specified as a child image. Specifically, in the captured image group U shown in FIG. 10, the telephoto image 110 captured between the parent images indicated by P1 and P2 is used as a child candidate image of a group having the wide-angle image of P1 as the parent image. Also good. As a result, the child image specifying unit 205 does not need to extract the feature points of all the captured images or all the telephoto images 110 in the captured image group U and check the correspondence relationship with the parent image. The telephoto image 110 captured between the time when the first image is captured and the time when the second group parent image is captured may be specified as a child image of the first group.

  The group generation unit 207 generates, for each of the shooting targets, a group composed of a parent image and a child image captured from the same shooting target, or a group composed of child images. The group generated by the group generation unit 207 is a group configured by a parent image and a child image for each shooting target, or a group configured by a child image for each shooting target. Specifically, when the subject to be imaged is the floor slab 6 of the case, a group of the wide-angle image 100 and the telephoto image 110 or the telephoto image 110 of each case is generated for each case.

<Grouping of photographed images>
Next, the captured image group U and the grouping result will be described using specific examples.

  FIG. 10 is a diagram showing a photographed image group U composed of photographed images photographed in the inspection work for the bridge 1. In FIG. 10, a part of a plurality of captured images constituting the captured image group U is illustrated. The photographed image group U is composed of a wide-angle image 100 that is a subject to be photographed, a telephoto image 110 of the floor slab 6, and a landmark image 112. The wide-angle image 100, the telephoto image 110, and the landmark image 112 are assigned “DSCF34OO” as an image number in the order of shooting, and are arranged in the order of shooting. In addition, each of the plurality of captured images constituting the captured image group U includes information regarding the time at which the captured image was captured, and information regarding capturing conditions (for example, focal length). Note that the wide-angle image 100 indicated by P1 in FIG. 10 is designated as the first group of parent images, and the wide-angle image 100 indicated by P2 is designated as the second group of parent images. The designation of the parent image is performed by, for example, displaying a thumbnail display of the captured image group U or the wide-angle image 100 on the display unit 326 and selecting the thumbnail display by the inspector.

  FIGS. 11 to 14 are diagrams showing the parent image received by the parent image receiving unit 203 and the child image specified by the child image specifying unit 205. 11 and 12 show the first group, and FIGS. 13 and 14 show the second group of parent images and child images.

  FIG. 11 is a diagram showing the wide-angle image 100 designated as the parent image of the first group. In this case, a plurality of wide-angle images obtained by separately shooting the shooting target are accepted as parent images. That is, the parent image receiving unit 203 receives the wide-angle image 100 captured by dividing the case into four as the first group parent image.

  FIG. 12 shows a telephoto image 110 specified as a child image of the first group in the photographed image group U. A child image is specified from the photographed image group U by the child image specifying unit 205 based on the image feature of the parent image and the image features of the telephoto image 110 specified in FIG.

  FIG. 13 shows a wide-angle image 100 designated as the parent image of the first group in the photographed image group U. In this case, the parent image receiving unit 203 receives the wide-angle image 100 captured by dividing the space into three as the second group parent image.

  FIG. 14 shows a telephoto image 110 specified as a child image of the first group in the photographed image group U. A child image is identified from the captured image group U based on the image feature of the parent image and the image features of the telephoto image 110 specified in FIG.

  FIG. 15 is a flowchart showing the operation of the image processing apparatus 200.

  First, the captured image group U is input to the image input unit 201 (image input step: step S10). The input of the captured image group U is performed via the wired communication unit 324, the mobile communication unit 318, the wireless LAN communication unit 320, or the short-range wireless communication unit 322 of the computer 300. Thereafter, the parent image receiving unit 203 receives the designation of the parent image of the first group in the wide-angle image 100 (parent image receiving step: step S12). For example, when the inspector selects the thumbnail display of the photographed image group U displayed on the display unit 326, the parent image is designated. Then, the child image specifying unit 205 specifies a child image in the telephoto image 110 (child image specifying step: step S14). For example, the child image specifying unit 205 detects corresponding points using a known image processing technique, calculates the matching between the image feature of the parent image and the image feature of the telephoto image 110 included in the photographed image group U, and has good matching Is identified as a child image. Thereafter, the group generation unit 207 generates a group composed of a parent image and a child image obtained by photographing the same photographing target or a group composed of the child images (group generation step: step S16).

  In the above embodiment, the hardware structure of a processing unit that executes various processes is the following various processors. The various processors are programmable processors that can change the circuit configuration after manufacturing, such as a CPU 310, which is a general-purpose processor that functions as various processing units by executing software (programs), and an FPGA (Field Programmable Gate Array). A dedicated electric circuit that is a processor having a circuit configuration designed exclusively for executing specific processing such as a logic device (PLD) and an application specific integrated circuit (ASIC) is included.

  One processing unit may be configured by one of these various processors, or may be configured by two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of CPU 310 and FPGA). May be. Further, the plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer 300 such as a client or server, one processor is configured with a combination of one or more CPUs 310 and software, There is a form in which this processor functions as a plurality of processing units. Secondly, as represented by System On Chip (SoC), etc., there is a form of using a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip. is there. As described above, various processing units are configured using one or more of the various processors as a hardware structure.

  Further, the hardware structure of these various processors is more specifically an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

<Second Embodiment>
Next, a second embodiment of the present invention will be described.

  FIG. 16 is a block diagram illustrating functions of the image processing apparatus 200 according to the present embodiment. The image processing apparatus 200 according to the present embodiment includes an image input unit 201, a parent image receiving unit 203, a child image specifying unit 205, a group generating unit 207, a display control unit 209, and a correction instruction receiving unit 211. In addition, the part demonstrated in FIG. 5 attaches | subjects the same code | symbol, and abbreviate | omits description.

  The display control unit 209 causes the display unit 326 to display a parent image and a child image for each group generated by the group generation unit 207. The display form of the group performed by the display control unit 209 is not particularly limited. For example, the display control unit 209 displays a thumbnail image of a parent image and a child image for each group.

  The correction instruction receiving unit 211 receives a child image correction instruction. For example, when the inspector selects a thumbnail image of a child image displayed on the display unit 326 by the display control unit 209, the correction instruction receiving unit 211 receives a child image correction instruction.

  The group generation unit 207 generates a group again based on the correction instruction received by the correction instruction reception unit 211. Specifically, the group generation unit 207 deletes or adds a child image from the group based on the correction instruction received by the correction instruction reception unit 211.

  17 and 18 are diagrams for explaining the correction of the child image. 17 shows a thumbnail of the child image specified by the child image specifying unit 205 of the first group on the touch panel display 302, and FIG. 18 shows a thumbnail of the child image specified by the child image specifying unit 205 of the second group. Is displayed on the touch panel display 302. The inspector confirms the thumbnail images on the touch panel display 302, determines that the images constituting the group are inappropriate, and selects them to exclude them from the group.

  In the case shown in FIG. 17, the mark image 112B is not inappropriate as the child image of the first group. That is, the inspector confirms the thumbnail image of the first group of child images displayed on the touch panel display 302, recognizes that the landmark image 112B is mixed, selects the landmark image 112B, and selects the correction instruction reception unit 211. Through the first group child image correction instruction. It should be noted that the mark image 112B has an extremely low correlation with the parent image of the first group (for example, the result of extracting the feature points and detecting the corresponding points), and the mark image 112B is extremely low. By informing the person in charge, the inspector may select an inappropriate image. Then, the correction instruction receiving unit 211 receives an instruction for selecting the mark image 112B, and the group generation unit 207 generates a new group excluding the mark image 112B.

  In the case shown in FIG. 18, the telephoto image 110M is inappropriate as a child image of the second group. For example, the telephoto image 110M is an image that has failed to be captured or an image that has been excessively captured. Therefore, the inspector confirms the thumbnail images of the second group of child images displayed on the touch panel display 302 and selects the telephoto image 110M. For example, when the image quality of the telephoto image 110M is not good, the correlation with the parent image of the second group is bad.

<Third Embodiment>
Next, a third embodiment of the present invention will be described.

  FIG. 19 is a block diagram illustrating functions of the image processing apparatus 200 according to the present embodiment. The image processing apparatus 200 of this embodiment includes an image input unit 201, a parent image receiving unit 203, a child image specifying unit 205, a group generating unit 207, a compositing target specifying unit 213, and a child image combining unit 215.

  The compositing target specifying unit 213 specifies a plurality of child images constituting the group as target images for image compositing. By grouping the grouping results of the captured image group U, effective grouping results can be used. For example, when the bridge 1 is inspected, the inspection report can be posted and the position of the damaged portion can be efficiently identified by compositing the telephoto image 110.

  The child image composition unit 215 performs image composition on the target image for image composition specified by the composition target specifying unit 213. That is, the child image composition unit 215 performs image composition on the child image specified as the composition target by the composition target specifying unit 213. Note that a known image composition technique is applied to the image composition performed by the child image composition unit 215.

  FIG. 20 is a diagram illustrating image composition of a plurality of grouped child images.

  The telephoto image 110 specified as a child image in the group generated by the group generation unit 207 is specified as a synthesis target by the synthesis target specification unit 213. The compositing target specifying unit 213 may set all of the grouped telephoto images 110 as a compositing target, or may select the telephoto image 110 received a designation from the inspector as a compositing target. Thereafter, the child image composition unit 215 performs composition of the child image (telephoto image 110) specified as the composition target. In the case shown in FIG. 20, by synthesizing the telephoto image 110 specified as the synthesis target, a synthesized image 115 in which the entire image of the photographing target (the floor slab 6) is captured is obtained. Note that the composite image 115 is not limited to the whole image of the subject to be photographed, and may be a composite image 115 in which a part of the subject is photographed.

  The example of the present invention has been described above, but the present invention is not limited to the above-described embodiment, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Bridge 2 Main girder 3 Horizontal girder 4 Counter-tilt structure 5 Horizontal composition 6 Floor slab 100 Wide-angle image 101A-101C Divided wide-angle image 110 Telephoto image 112 Mark image 115 Composite image 200 Image processing apparatus 201 Image input part 203 Parent image reception part 205 Child Image identification unit 207 Group generation unit 209 Display control unit 211 Correction instruction reception unit 213 Composition target identification unit 215 Child image synthesis unit 300 Computer 301 Case 302 Touch panel display 303 Operation button 304 Speaker 305 Built-in camera 306 External connection terminal 310 CPU
312 System bus 314 Main memory 316 Non-volatile memory 318 Mobile communication unit 318A Antenna 320 Wireless LAN communication unit 320A Antenna 322 Short-range wireless communication unit 322A Antenna 324 Wired communication unit 326 Display unit 328 Input unit 330 Key input unit 332 Audio processing unit 334 Image Processing Unit Step S10 to Step S16 Image Processing Method

Claims (12)

An image processing apparatus that groups a group of captured images including captured images of a plurality of shooting targets into groups for each of the shooting targets,
The photographed image group including one or a plurality of first images photographed for each of the photographing objects and a plurality of second images photographed second under photographing conditions different from the first photographing is input. An image input unit,
A parent image receiving unit that receives one of the first images as a parent image representing the whole image of the shooting target;
A child image specifying unit for specifying, as a child image, the second image obtained by shooting a part of the shooting target of the parent image based on the received image feature of the parent image and the image feature of the second image; ,
For each of the shooting targets, a group formed by the parent image and the child image that shot the same shooting target, or a group generation unit that generates a group formed by the child image,
An image processing apparatus comprising: The parent image receiving unit receives a plurality of the first images obtained by dividing and shooting the shooting target as the parent image,
The child image specifying unit combines the plurality of first images received as the parent image into one image representing the whole image of the shooting target, and includes image characteristics of the one image. The image processing apparatus according to claim 1, wherein the child image is specified based on an image feature of the second image.   The said child image specific | specification part extracts the feature point of the said 2nd image and the said parent image, and specifies the said child image based on the correspondence of the extracted said feature point. Image processing device. Each of the plurality of photographed images constituting the photographed image group has information regarding the time of photographing,
The child image specifying unit sets the second image taken after or before the time when the parent image of one group is taken as a child candidate image of the child image, and determines the child image from the child candidate image. The image processing apparatus according to claim 1, wherein an image is specified. Each of the plurality of photographed images constituting the photographed image group has information regarding the time of photographing,
The child image specifying unit may include the second image captured between the time when the parent image of the first group is captured and the time when the parent image of the second group is captured. The image processing apparatus according to claim 1, wherein the child image is a child candidate image of the child image, and the child image is specified from the child candidate image. Each of the plurality of photographed images constituting the photographed image group has information related to the photographing conditions,
6. The parent image receiving unit extracts the first image from the photographed image group based on the photographing condition, and accepts the parent image from the extracted first image. The image processing apparatus according to any one of the above. The photographed image group includes a landmark image indicating that the photographing object of the photographed image is changed to another photographing object,
Each of the plurality of photographed images and landmark images constituting the photographed image group has information regarding the time of photographing,
7. The image processing apparatus according to claim 1, wherein the parent image receiving unit receives the parent image for the first image taken immediately after or immediately before the landmark image is taken. A display control unit that causes the display unit to display the parent image and the child image for each group generated by the group generation unit;
A correction instruction receiving unit that receives a correction instruction for the child image,
The image processing apparatus according to claim 1, wherein the group generation unit generates the group again based on the correction instruction.   9. The image processing apparatus according to claim 1, further comprising: a compositing target specifying unit that specifies the plurality of child images constituting the group as target images for image compositing.   The image processing apparatus according to claim 9, further comprising a child image synthesis unit that synthesizes the target image of the image synthesis specified by the synthesis target specifying unit. An image processing method for grouping photographed image groups including photographed images of a plurality of photographing objects into groups for each of the photographing objects,
The photographed image group including one or a plurality of first images photographed for each of the photographing objects and a plurality of second images photographed second under photographing conditions different from the first photographing is input. An image input step,
A parent image receiving step of receiving any one of the first images as a parent image representing the whole image of the photographing target;
A child image specifying step of specifying, as a child image, the second image obtained by shooting a part of the shooting target of the parent image based on the received image feature of the parent image and the image feature of the second image; ,
A group generation step of generating a group composed of the parent image and the child image obtained by photographing the same photographing object for each of the photographing objects, or a group composed of the child images;
An image processing method including: A program for causing a computer to execute an image processing method for grouping captured image groups including captured images of a plurality of capturing targets into groups for each of the capturing targets,
The photographed image group including one or a plurality of first images photographed for each of the photographing objects and a plurality of second images photographed second under photographing conditions different from the first photographing is input. An image input step,
A parent image receiving step of receiving any one of the first images as a parent image representing the whole image of the photographing target;
A child image specifying step of specifying, as a child image, the second image obtained by shooting a part of the shooting target of the parent image based on the received image feature of the parent image and the image feature of the second image; ,
A group generation step of generating a group composed of the parent image and the child image obtained by photographing the same photographing object for each of the photographing objects, or a group composed of the child images;
A program for causing a computer to execute an image processing method including:

Images