JP2021103126A - Information processing device, photographing support method and photographing support program - Google Patents

Abstract

To determine whether an image detective area occurs in a combined image of an inspection object in a photographing process.SOLUTION: An information processing device 1 is used in a photographing process for photographing an inspection object in state change analysis for detecting state change of the inspection object by analyzing a combined image obtained by combining a plurality of images obtained by photographing the inspection object. The information processing device 1 includes an acquisition part 32 for acquiring a plurality of images acquired by two-dimensionally or three-dimensionally moving a photographing device so as to generate an overlapping area in a photographing range with respect to the inspection object as an image group, and a determination part 33 for using images included in the image group acquired by the acquisition part 32 to determine whether an image detective area occurs in a combined image generated from the image group.SELECTED DRAWING: Figure 8

Description

The present invention relates to an information processing device, a shooting support method, and a shooting support program.

Conventionally, in the inspection of structures such as concrete walls such as dams, bridges and tunnels and concrete floors such as buildings, the inspection target is imaged with a camera and the acquired image is analyzed to detect deformation such as cracks. Is being done (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2019-14238

When the object to be inspected is large, the resolution of one image is not sufficient, and it is not possible to detect deformation with sufficient accuracy. Therefore, for example, a large number of high-definition images are acquired by performing imaging while moving the camera so that a superposed region is generated at the angle of view, and image analysis is performed using the combined image obtained by combining the acquired images. Deformation is being detected.

The image combination process is performed by combining images that are adjacent to each other. However, when the overlapping region of adjacent images is small, sufficient information for image combination cannot be obtained, and image combination may not be possible. In this case, an image defect region is generated in the combined image representing the entire structure.

In particular, for relatively large structures such as dams and bridges, drones may be used for shooting, and stable propulsion control is difficult due to the influence of wind, etc., so it is necessary to acquire images in the planned flight course. There is a problem that it is difficult. In such a case, there is a higher possibility that an image defect region will occur in the combined image.

When the structure to be inspected is a dam, a bridge, or the like, the shooting process of taking a picture and the image joining process of performing an image joining process based on the acquired image are generally set up on different schedules. Therefore, if an image defect region is generated in the combined image in the image combination process, the image of the structure corresponding to the image defect region must be acquired again, and it is necessary to readjust the shooting schedule or the like. Therefore, the man-hours required for defect analysis will increase.

The present invention has been made in view of such circumstances, and is an information processing device and an imaging support capable of avoiding the occurrence of an image defect region in a combined image used for image analysis for detecting deformation. The purpose is to provide methods and photography support programs.

The first aspect of the present invention is in the process of photographing the inspection target in the deformation analysis of detecting the deformation of the inspection target by analyzing the combined image obtained by combining a plurality of images of the inspection target. The information processing apparatus used, which acquires a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so as to generate a superposed region in the imaging range with respect to the inspection target as an image group. Information processing including an acquisition means and a determination means for determining whether or not an image defect region occurs in a combined image generated from the image group by using an image included in the image group acquired by the acquisition means. It is a device.

In the second aspect of the present invention, the inspection target is subjected to the first image combination processing of combining a plurality of images of the inspection target, and the combined image generated by the first image combination process is analyzed. An information processing device used in the imaging process of photographing the inspection target in the deformation analysis for detecting the deformation, and the imaging means is set to be two-dimensional or tertiary so that an overlapping region is generated in the imaging range with respect to the inspection target. An acquisition means for acquiring a plurality of images originally acquired by moving them as an image group and a second image combination process in which the amount of processing is smaller than that of the first image combination process are executed and acquired by the acquisition means. It is an information processing apparatus including a combined image generation means for generating a combined image from an image included in the image group, and a display means for displaying the generated combined image.

A third aspect of the present invention is in the process of photographing the inspection target in the deformation analysis for detecting the deformation of the inspection target by analyzing the combined image obtained by combining a plurality of images of the inspection target. This is an image shooting support method to be executed, and a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so as to generate a superposed area in the photographing range are acquired as an image group with respect to the inspection target. This is a photographing support method in which a computer executes a step of determining whether or not an image defect region occurs in a combined image generated from the image group by using the image included in the image group.

A fourth aspect of the present invention is to execute a first image combination process of combining a plurality of images of an inspection target, and analyze the combined image generated by the first image combination process to obtain the inspection target. In the deformation analysis for detecting deformation, it is a shooting support method executed in the shooting process of shooting the inspection target, and the imaging means is two-dimensional or the imaging means is set so as to generate an overlapping region in the shooting range with respect to the inspection target. A step of acquiring a plurality of images acquired by three-dimensionally moving them as an image group and a second image combining process in which the amount of processing is smaller than that of the first image combining process are executed and included in the image group. This is a shooting support method in which a computer executes a step of generating a combined image from an image and a step of displaying the generated combined image.

A fifth aspect of the present invention is in a deformation analysis for detecting a deformation of the inspection target by analyzing a combined image obtained by combining a plurality of images of the inspection target, in a photographing process of photographing the inspection target. An image shooting support program to be executed, in which a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so as to generate a superposed area in the photographing range are acquired as an image group with respect to the inspection target. This is a shooting support program for causing a computer to perform a process of performing the process and a process of determining whether or not an image defect region is generated in the combined image generated from the image group by using the image included in the image group. ..

A sixth aspect of the present invention is to execute a first image combination process of combining a plurality of images of an inspection target, and analyze the combined image generated by the first image combination process to obtain the inspection target. In the deformation analysis for detecting deformation, it is a shooting support program executed in the shooting process of shooting the inspection target, and the imaging means is set in two dimensions or the imaging means is set so that an overlapping region is generated in the shooting range with respect to the inspection target. A process of acquiring a plurality of images acquired by moving three-dimensionally as an image group and a second image combination process in which the amount of processing is smaller than that of the first image combination process are executed and included in the image group. This is a shooting support program for causing a computer to execute a process of generating a combined image from an image and a process of displaying the generated combined image.

According to the present invention, it is possible to prevent an image defect region from being generated in a combined image used for image analysis for detecting deformation.

It is a figure which showed the schematic process of the deformation analysis method which concerns on 1st Embodiment of this invention. It is a figure which showed typically an example of the inspection object (structure) which concerns on 1st Embodiment of this invention, and the movement path set in advance with respect to the inspection object. It is a figure which showed typically an example of the angle of view of the image acquired when the image is taken along the preset movement path in the image-taking process which concerns on 1st Embodiment of this invention. It is a figure which showed the hardware configuration of the information processing apparatus which concerns on 1st Embodiment of this invention. It is a figure which showed typically the case where the actual movement locus deviates from the preset movement path. It is a figure which showed typically the angle of view when the actual movement locus deviates from the preset movement path. It is a figure which showed typically an example of the image defect region generated in the combined image. It is a functional block diagram which showed the photographing support function provided in the information processing apparatus which concerns on 1st Embodiment of this invention. It is a figure which showed typically an example of the processing target area which concerns on 1st Embodiment of this invention. It is a flowchart which showed an example of the processing procedure of the photographing support method which concerns on 1st Embodiment of this invention. It is a flowchart which showed the other example of the processing procedure of the photographing support method which concerns on 1st Embodiment of this invention. It is a functional block diagram which showed the photographing support function provided in the information processing apparatus which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating the processing content realized by the specific part which concerns on 2nd Embodiment of this invention. It is a flowchart which showed an example of the processing procedure of the photographing support method which concerns on 2nd Embodiment of this invention. It is a flowchart which showed an example of the processing procedure of the photographing support method which concerns on 2nd Embodiment of this invention. It is a functional block diagram which showed the photographing support function provided in the information processing apparatus which concerns on 3rd Embodiment of this invention. It is a flowchart which showed an example of the processing procedure of the photographing support method which concerns on 3rd Embodiment of this invention. It is a flowchart which showed an example of the processing procedure of the photographing support method which concerns on 3rd Embodiment of this invention.

[First Embodiment]
Hereinafter, the information processing apparatus, the photographing support method, and the photographing support program according to the first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing a schematic process of a deformation analysis method according to the present embodiment. As shown in FIG. 1, the deformation analysis method includes a photographing process (SA1), a combined image generation process (SA2), an image analysis process (SA3), and an analysis result output process (SA4).

In the shooting process, the structure to be inspected is photographed. For shooting, for example, an imaging device such as a camera is moved along a preset movement path to acquire a large number of images. The image acquired in the shooting process is once taken into the information processing device 1 (see FIG. 4), subjected to predetermined preprocessing here, and then recorded on a cloud server or a recording medium, and the combined image generation process in the subsequent stage. It is used in. The details of the shooting process will be described later.

In the combined image generation process, a plurality of images acquired in the shooting process are combined to generate a combined image. The larger the structure to be inspected, the larger the number of images acquired in the shooting process, and the longer the time required to generate the combined image. For example, the processing may be performed for several days.
Since the combined image generation process is performed at a place different from the imaging process, for example, the information processing device used in the combined image generation process is different from the information processing device 1 (see FIG. 4) used in the imaging process, for example. It is said to be a terminal.
The combined image generation process is realized by the information processing apparatus executing an image combining program for the image combining process (first image combining process).

In the image analysis process, deformation of the structure is detected by image analysis of the combined image generated in the combined image generation process. The image analysis process is realized by the information processing apparatus executing an image analysis program.

In the analysis result output process, the analysis result obtained in the image analysis process, that is, the deformation analysis result is displayed on a display device or the like, so that the worker or the like is notified of the deformation analysis result.

The combined image generation process, the image analysis process, and the analysis result output process can be realized by appropriately adopting known techniques.

Next, the above-mentioned photographing process (SA1) will be described in more detail with reference to the drawings.
FIG. 2 is a diagram schematically showing an example of an inspection target (structure) and a movement route preset for the inspection target. Here, the inspection target is a structure, and examples thereof include concrete walls such as dams, bridges, and tunnels, and concrete floors such as buildings. As shown in FIG. 2, in the photographing process, an imaging device such as a camera is moved along a movement path set in advance with respect to the inspection target, and a large number of images are acquired. At this time, as shown in FIG. 3, the image is acquired so that a superposed region is generated in the shooting range (angle of view) in the overlapping direction which is the moving direction and the side wrap direction which is the direction orthogonal to the moving direction. ..
Shooting may be performed, for example, by moving a self-propelled robot equipped with a camera two-dimensionally along a moving path, or by manipulating a drone equipped with a camera two-dimensionally along the moving path. It may be done by moving the target or three-dimensionally.

The image acquired by shooting is associated with the identification number (file number), shooting time, etc., and is stored in, for example, a recording medium in the camera. Then, when the amount of remaining data on the recording medium falls below a predetermined value, the data is replaced with a new recording medium and shooting is continued.
Further, the recording medium on which the image is recorded is connected to the information processing device 1 (see FIG. 4) used in the photographing process, so that the image recorded on the recording medium is taken into the information processing device 1. Hereinafter, the image group recorded on one recording medium is referred to as a "sub-image group", and the area to be inspected corresponding to the "sub-image group" is referred to as a "sub-area". In other words, the "sub-region" corresponds to a part of the entire inspection target, and the "sub-image group" is composed of a plurality of images corresponding to a part of the entire inspection target. Then, an image of the entire inspection target is composed of a plurality of sub-image groups.

The captured sub-image group is stored in one folder in the storage unit 31 (see FIG. 8) in the information processing device 1, for example. In this way, the storage unit 31 of the information processing device 1 stores the images acquired by the image pickup device such as a camera for each sub-image group.

The image acquisition method in the information processing device 1 is not limited to the above example. For example, images are sequentially transmitted from an image pickup device such as a camera to the information processing device 1 by wireless communication, and the image is sequentially transmitted to the information processing device 1 in the information processing device 1. The configuration may be such that it is stored in the storage unit 31 (see FIG. 8). In this case, the plurality of images sequentially input may be set as a sub-image group for each predetermined number of images and stored in the storage unit 31.

FIG. 4 is a diagram showing a hardware configuration of the information processing device 1 according to the present embodiment. As shown in FIG. 4, the information processing device 1 according to the present embodiment includes, for example, a CPU 11, an auxiliary storage device 12 for storing a program executed by the CPU 11, and a main memory functioning as a work area at the time of executing each program. It includes a device 13, a communication interface 14 for connecting to a network, an input unit 15 including a keyboard, a mouse, and the like, and a display unit 16 such as a liquid crystal display device for displaying data. Each of these parts is connected via, for example, a bus 18. Examples of the auxiliary storage device 12 include magnetic disks, magneto-optical disks, semiconductor memories, and the like.

The information processing device 1 according to the present embodiment has a shooting support function.
For example, the above-mentioned photography of the inspection target may be affected by the surrounding environment. For example, when shooting with a drone, the actual movement trajectory deviates from the preset movement path as shown in FIG. 5 due to the influence of wind and the difficulty of maneuvering. There is a risk. In such a case, for example, as shown in FIG. 6, the angle of view (shooting range) of the actually acquired image deviates greatly from the planned angle of view, and a sufficiently superimposed area of adjacent images can be obtained. It may not be possible. In particular, this reduction in the overlapping region tends to occur in the direction intersecting the overlapping direction. Then, when such a deviation of the movement locus occurs, as shown in FIG. 7, there is a possibility that an image defect region may occur in the combined image in the combined image generation process performed later.

Therefore, the information processing apparatus 1 according to the present embodiment determines in the imaging process whether or not an image defect region is generated in the combined image generated from the image based on the image acquired by the image pickup apparatus, and determines whether or not an image defect region is generated, and the image defect. When it is determined that an area is generated, a re-shooting area including the image defect area is set, and the re-shooting area is notified to the worker. In this way, by notifying the worker of the image defect area at the shooting stage, it is possible to prevent the image defect area from being generated in the combined image in the subsequent combined image generation process, and it is not necessary to make adjustments such as re-shooting. can do.

Hereinafter, the photographing support function included in the information processing apparatus 1 according to the present embodiment will be described with reference to the drawings. FIG. 8 is a functional configuration diagram showing a photographing support function included in the information processing device 1 according to the present embodiment. As shown in FIG. 8, the information processing device 1 includes a storage unit 31, an acquisition unit 32, a determination unit 33, and a display unit 16. The function realized by each of these parts is realized, for example, by the CPU 11 reading the image defect determination program stored in the auxiliary storage device 12 into the main storage device 13 and executing the program.

The storage unit 31 stores a plurality of images continuously acquired by moving the image pickup device two-dimensionally or three-dimensionally so that a superposed area is generated in the shooting range (angle of view) with respect to the inspection target. ing. As described above, in the storage unit 31 according to the present embodiment, as an example, a plurality of images are stored in folders for each sub-image group.

The acquisition unit 32 acquires a plurality of images for each sub-image group from the storage unit 31, for example.
The determination unit 33 uses a plurality of images for each sub-image group to determine whether or not an image defect region occurs in the combined image generated from the sub-image group. For example, the determination unit 33 includes a setting unit 41, an extraction unit 42, and a combined image generation unit 43.

For example, as shown in FIG. 9, the setting unit 41 sets a plurality of processing target areas in a direction intersecting the overlapping direction in the sub area corresponding to the sub image group.
The extraction unit 42 extracts a plurality of images corresponding to the processing target area from the sub-image group for each processing target area. Here, if the various conditions at the time of shooting, for example, the angle of view, the superposition ratio (overlapping amount) of adjacent images in the overlapping direction, and the movement path are known, the image corresponding to which position in the movement path each image is known. This makes it possible to specify at which position the image is in the entire inspection target (sub-region). The angle of view can be specified from the focal length of the camera and the number of pixels. The extraction unit 42 extracts a plurality of images corresponding to the processing target area from the sub-image group based on the above conditions.

The combined image generation unit 43 generates a combined image of the processing target region using a plurality of images extracted by the extraction unit 42. The combined image generation unit 43 generates a combined image by, for example, executing a known image combining process (second image combining process). The image combination process adopted by the combined image generation unit 43 is not particularly limited, but the amount of processing is smaller than that of the image combination process (first image combination process) used in the combined image generation process (SA2) to be performed later. In other words, it is preferable to use an image combination process having low combination accuracy.

The determination unit 33 determines whether or not an image defect region is generated in the combined image generated by the combined image generation unit 43, and causes the display unit 16 to display the determination result. For example, when an image defect region is generated, the determination unit 33 causes the display unit 16 to display the position of the image defect region with respect to the entire inspection target. Further, at this time, the determination unit 33 may set the re-photographing area including the image defect area, and display the display image indicating the re-photographing area in the inspection target on the display unit 16. As a result, the worker can easily grasp which area the shooting should be performed again, and the shooting can be efficiently performed again.

Next, the photographing support method according to the present embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing an example of the processing procedure of the shooting support method according to the present embodiment.
As shown in FIG. 10, first, a plurality of images corresponding to the unprocessed sub-image group are acquired from the storage unit 31 (SB1), and before the images satisfying a predetermined condition are excluded from the acquired images of the sub-image group. Perform processing (SB2). Here, the "image satisfying a predetermined condition" is, for example, an out-of-focus image, an image containing foreign matter (for example, a worker's finger, etc.), halation (overexposure), an image having shadows, or the like. In the preprocessing, images satisfying such a predetermined condition are automatically or manually identified, and these images are excluded.

Subsequently, in the sub-region corresponding to the sub-image group, as shown in FIG. 9, a plurality of processing target regions are set in the direction intersecting the overlapping direction (SB3). Subsequently, a plurality of images corresponding to the unprocessed processing target area are extracted from the pre-processed sub-image group (SB4). Subsequently, a combined image is generated using the extracted plurality of images (SB5).

Subsequently, it is determined whether or not the combined image has been generated for all the processing target areas (SB6), and if it has not been generated (SB6: NO), the process returns to step SB4 and the combined image has not been generated yet. A plurality of images corresponding to the processing target area of the processing are extracted. Then, when the combined images are generated for all the processing target regions by repeating the processing of steps SB4 to SB5 (SB6: YES), it is determined whether or not an image defect region is generated in each combined image. The determination is made (SB7), and the determination result is displayed on the display screen (SB8).

The display example of the determination result is not particularly limited, but for example, when the image defect area does not occur, a message indicating that no error has occurred is displayed in the sub area, and the image defect area occurs. A display screen is displayed so that the position of the image defect area in the sub area can be known. At this time, information indicating which area of the entire inspection target the sub-area corresponds to may also be displayed. Further, a re-photographing area including an image defect area in the sub-area may be set, and a display screen may be displayed on the display unit 16 so that the re-photographing area of the entire inspection target can be seen.

Subsequently, it is determined whether or not the image defect determination has been performed for all the sub-regions (SB9), and if the image defect determination has not been completed for all the sub-regions (SB9: NO), step SB1 is performed. Return, acquire the sub-image group corresponding to the undetermined sub-area, and repeat the subsequent processing. Then, in step SB9, when the image defect determination is performed for all the sub-regions (SB9: YES), the process ends.

As described above, according to the information processing apparatus, the shooting support method, and the shooting support program according to the present embodiment, it is determined whether or not an image defect region occurs in the combined image in the shooting process. It is possible to notify the worker in the shooting process whether or not there is a possibility that an inspection area has occurred. As a result, it is possible to prevent an image defect region from being generated in the combined image generated in the combined image generation process performed after the photographing process. As a result, it is possible to eliminate the need for readjustment for photographing, and it is possible to efficiently perform deformation analysis.

Further, according to the information processing device, the shooting support method, and the shooting support program according to the present embodiment, the display unit 16 displays the determination result so that the worker can confirm the position of the image defect region. Therefore, the worker who has confirmed this display screen can grasp which area of the inspection target should be photographed again. This makes it possible to efficiently perform re-shooting. Further, in this way, when the image is photographed again, it may be determined whether or not the image defect region is generated again with respect to the image group obtained by the re-imaging. This makes it possible to reliably generate a combined image for deformation analysis in a later process.

In this embodiment, as shown in FIG. 11, after the preprocessing performed in step SB2, a size reduction process for further reducing the number of pixels may be performed (SB2'). By reducing the number of pixels in this way, it is possible to reduce the subsequent processing, and it is possible to shorten the processing time required for the image defect determination processing.
The size reduction process for reducing the number of pixels may be performed at any timing after step SB1 and before step SB5, for example, between steps SB3 and SB4, or between steps SB4 and SB5. You may do it.

Further, in the present embodiment, the determination result by the determination unit 33 is displayed on the display unit 16, but instead of the determination result, the combined image generated by the combined image generation unit 43 is displayed on the display unit 16. May be. That is, in this case, it is not necessary to determine whether or not an image defect region is generated based on the combined image generated by the combined image generation unit 43.
By displaying the combined image on the display unit 16 in this way, the worker who has confirmed the combined image can confirm in which region the image defect has occurred. This makes it possible to inform the worker that re-shooting is necessary.

Further, in the present embodiment, the combined image generated by the combined image generation unit 43 is set as the target processing region in the sub-region, but the region for generating the combined image is not limited to this example. For example, a combined image may be generated using all the images in the sub-image group. Further, the present invention is not limited to this example, and for example, a combined image may be generated using the image of the entire inspection target after the photographing of the entire inspection target is completed. In this case, it takes more time than generating a combined image for each sub-image group, but the image combining process (first image combining process) used in the combined image generation process (SA2 in FIG. 1) to be performed later. By performing image combination using an image combination program with a smaller amount of processing than the above, it is possible to obtain a combined image in a shorter time than the combined image generation process in the combined image generation process.
Further, in this case as well, the combined image may be generated using an image that has been subjected to the size reduction processing that reduces the number of pixels. In this case, the time required for generating the combined image is further shortened. Is possible.

[Second Embodiment]
Next, the information processing device, the shooting support method, and the shooting support program according to the second embodiment of the present invention will be described with reference to the drawings.
In the above-described first embodiment, the combined image is generated and the image defect determination is performed, but this embodiment is different from the above-mentioned first embodiment in that the image defect determination is performed without generating the combined image. Hereinafter, the same configurations as those of the first embodiment described above will be assigned the same reference numerals to the present embodiment, and the description thereof will be omitted, and the differences will be mainly described.

FIG. 12 is a functional configuration diagram showing a photographing support function included in the information processing device 1a according to the present embodiment. As shown in FIG. 12, the information processing device 1a includes a setting unit 41, an extraction unit 42, and a specific unit 44 as the determination unit 33a. That is, the specific unit 44 is provided in place of the combined image generation unit 43 according to the first embodiment described above.

The specific unit 44 extracts two images adjacent to each other in the side wrap direction as a set image from a plurality of images corresponding to the processing target area extracted by the extraction unit 42. Then, for example, as shown in FIG. 13, feature points corresponding to each other in the set image are specified as corresponding points. Then, when the number of the specified corresponding points (3 points in the case of FIG. 13) is equal to or less than a predetermined threshold value, the region of the set image is specified as the image defect region.

Then, the specific unit 44 performs the above processing on all the set images in the processing target area, so that it is determined whether or not the image is a defective area according to the number of corresponding points, and the determination result is displayed on the display unit 16. Is displayed. The example of the display screen is the same as that of the first embodiment.

Next, the photographing support method according to the present embodiment will be described with reference to FIGS. 14 and 15. 14 and 15 are flowcharts showing an example of the processing procedure of the shooting support method according to the present embodiment.
As shown in FIG. 14, first, a plurality of images corresponding to the unprocessed sub-image group are acquired from the storage unit 31 (SC1), and before the images satisfying a predetermined condition are excluded from the acquired images of the sub-image group. Perform processing (SC2). Subsequently, in the sub-area corresponding to the sub-image group, as shown in FIG. 9, a plurality of processing target areas are set in the direction intersecting the overlapping direction (SC3), and the sub-image group after the preprocessing is not yet processed. A plurality of images corresponding to the processing target area of the processing are extracted (SC4). As described above, SC1 to SC4 are processed in the same manner as SB1 to SB4 according to the first embodiment.

Subsequently, two adjacent images are extracted as a set image from the extracted plurality of images (SC5), and feature points corresponding to each other in the set image are specified as corresponding points (SC6). Next, it is determined whether or not the number of corresponding points is equal to or less than a predetermined threshold value (SC7). This threshold is set to the number of corresponding points to the extent that a combined image can be generated. That is, when the number of corresponding points exceeds the threshold value, it can be considered that the combined image can be generated, and when it is less than the threshold value, the combined image cannot be generated. As a result, when the number of corresponding points is equal to or less than the threshold value (SC7: YES), the region of the set image is specified as the image defect region (SC8). On the other hand, when the number of corresponding points exceeds the threshold value, it is determined that the region is not an image defect region, and the process proceeds to step SC9.

In step SC9, it is determined whether or not the corresponding points have been specified for all the group images in the processing target area, and if the corresponding points have not been specified for all the group images, the process returns to step SC5 and is not yet performed. Extract the set image of processing. Then, when the corresponding points are specified and determined for all the set images in the processing target area by repeating the processing from step SC5 to step SC9 (SC9: YES), the processing is performed in all the processing target areas. It is determined whether or not it has finished (SC10). As a result, if the processing is not completed for all the processing target areas (SC10: NO), the process returns to step SC4, and the same processing is performed for the unprocessed processing target areas. As a result, in step SC10, when it is determined that the processing has been completed for all the processing targets (SC10: YES), the determination result is displayed on the display unit 16 (SC11).

As a result, for example, when the image defect region does not occur, a message indicating that no error has occurred in the sub region is displayed, and when the image defect region occurs, the image defect in the sub region is displayed. A display screen is displayed so that the area (the area specified as the image defect area in step SC8) can be seen. At this time, information indicating which area of the entire inspection target the sub-area corresponds to may be displayed together. Further, a re-photographing area including an image defect area in the sub-area may be set, and a display screen may be displayed on the display unit 16 so that the re-photographing area in the entire inspection target can be seen.

Subsequently, it is determined whether or not the image defect determination has been performed for all the sub-regions (SC12), and if the image defect determination has not been completed for all the sub-regions (SC12: NO), FIG. Returning to step SC1, the sub-image group corresponding to the undetermined sub-region is acquired, and the subsequent processing is repeated. Then, in step SC12, when it is determined that the image defect determination has been performed for all the sub-regions (SC12: YES), the process ends.

As described above, according to the information processing device, the shooting support method, and the shooting support program according to the present embodiment, whether or not the region is an image defect region based on the number of corresponding points in the group image without generating a combined image. Therefore, the time required for the image defect determination process can be shortened as compared with the case of generating the combined image.

In the present embodiment, it is determined whether or not a defective image region is generated for each sub-image group, but the present invention is not limited to this example. For example, instead of making a judgment for each sub-image group, whether or not a defective image area is generated for an image group including a plurality of images corresponding to the entire inspection target after all the images to be inspected have been photographed. May be determined.

Further, also in the present embodiment, the size reduction process for reducing the number of pixels may be performed at an arbitrary timing between steps SC1 and SC6. By reducing the number of pixels in this way, it is possible to reduce the subsequent processing, and it is possible to further shorten the processing time required for the image defect determination processing.

[Third Embodiment]
Next, the information processing device, the shooting support method, and the shooting support program according to the third embodiment of the present invention will be described with reference to the drawings.
In the second embodiment described above, feature points corresponding to each other in the set image are specified as corresponding points, and image defect determination is performed based on the number of corresponding points. However, in the present embodiment, the degree of similarity of the images is determined. It differs from the first embodiment and the second embodiment in that the image defect is determined based on the above.
Hereinafter, the same configurations as those of the first embodiment and the second embodiment described above will be designated by the same reference numerals, and the description thereof will be omitted, and the differences will be mainly described.

FIG. 16 is a functional configuration diagram showing a photographing support function included in the information processing apparatus 1b according to the present embodiment. As shown in FIG. 16, the information processing device 1b includes a similar image identification unit 45 as a determination unit 33b.

The similar image identification unit 45 sequentially identifies a target image from a plurality of images corresponding to the processing target area extracted by the extraction unit 42, and a plurality of images in the processing target area are similar to the specified target image. Detect from inside. The similar image means, for example, an image having a degree of similarity equal to or higher than a predetermined threshold value. As a result, when an image having a similarity equal to or higher than a predetermined threshold value cannot be detected, the target image and its peripheral region are specified as an image defect region. Then, by sequentially performing such processing on each image in the processing target area, the image defect area in the processing target area is determined.

Then, when the image defect determination in all the processing target areas is completed, the determination result is displayed on the display unit 16. The example of the display screen is the same as that of the first embodiment.

Next, the photographing support method according to the present embodiment will be described with reference to FIGS. 17 and 18. 17 and 18 are flowcharts showing an example of the processing procedure of the shooting support method according to the present embodiment.
As shown in FIG. 17, first, a plurality of images corresponding to the sub image group are acquired from the storage unit 31 (SD1), and preprocessing is performed to exclude an image satisfying a predetermined condition from the acquired images of the sub image group. (SD2). Subsequently, in the sub-area corresponding to the sub-image group, as shown in FIG. 6, a plurality of processing target areas are set in the direction intersecting the overlapping direction (SD3), and the sub-image group after the preprocessing is not yet processed. A plurality of images corresponding to the processing target area of the processing are extracted (SD4). As described above, SD1 to SD4 are the same processes as SB1 to SB4 according to the first embodiment.

Subsequently, one target image to be processed is specified from the extracted plurality of images (SD5), and an image similar to the specified target image is detected from the plurality of images extracted in step SD4 (SD6). .. Subsequently, it is determined whether or not a similar image is detected (SD7), and if it is not detected (SD7: NO), a predetermined region including the target image is specified as an image defect region (SD8). On the other hand, when a similar image is detected (SD7: YES), it is determined that the region is not an image defect region, and the process proceeds to step SD9.

In step SD9, it is determined whether or not similar image detection processing has been performed for all images in the processing target area, and if not all images have been detected (SD9: NO), the process returns to step SD5 and is not processed. The image of is specified as the target image. Then, when the similar image detection processing is performed for all the images in the processing target area by repeating the processing from step SD5 to step SD9 (SD9: YES), is the processing completed in all the processing target areas? Whether or not it is determined (SD10 in FIG. 18).

As a result, if the processing is not completed for all the processing target areas (SD10: NO), the process returns to step SD4 of FIG. 17, and the same processing is performed for the unprocessed processing target areas. As a result, in step SD10, when it is determined that the processing has been completed for all the processing target areas (SD10: YES), the determination result is displayed on the display unit 16 (SD11).

As a result, for example, when the image defect region does not occur, a message indicating that no error has occurred in the sub region is displayed, and when the image defect region occurs, the image defect in the sub region is displayed. A display screen is displayed so that the position of the region (the region specified as the image defect region in step SD8) can be seen. At this time, information indicating which area of the entire inspection target the sub-area corresponds to may be displayed together. Further, a re-photographing area including an image defect area in the sub-area may be set, and a display screen may be displayed on the display unit 16 so that the re-photographing area of the entire inspection target can be seen.

Subsequently, it is determined whether or not the image defect determination has been performed for all the sub-regions (SD12), and if the image defect determination has not been completed for all the sub-regions (SD12: NO), FIG. Returning to step SD1, the sub-image group corresponding to the undetermined sub-region is acquired, and the subsequent processing is repeated. Then, in step SD12, when it is determined that the image defect determination has been performed for all the sub-regions (SD12: YES), the process ends.

As described above, according to the information processing device, the shooting support method, and the shooting support program according to the present embodiment, whether or not the region is an image defect region based on the number of corresponding points in the group image without generating a combined image. Therefore, the time required for the image defect determination process can be shortened as compared with the case of generating the combined image.
In the present embodiment, it is determined whether or not a defective image region is generated for each sub-image group, but the present invention is not limited to this example. For example, instead of making a judgment for each sub-image group, whether or not a defective image area is generated for an image group including a plurality of images corresponding to the entire inspection target after all the images to be inspected have been photographed. May be determined.

Further, also in the present embodiment, the size reduction process for reducing the number of pixels may be performed at an arbitrary timing between step SD1 and step SD6. By reducing the number of pixels in this way, it is possible to reduce the subsequent processing, and it is possible to further shorten the processing time required for the image defect determination processing.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or improvements can be made to the above embodiments without departing from the gist of the invention, and the modified or improved forms are also included in the technical scope of the present invention. In addition, the above embodiments may be combined as appropriate.
Further, the processing procedure in each flowchart described in the above embodiment is also an example, and unnecessary steps are deleted, new steps are added, or the processing order is changed within a range that does not deviate from the gist of the present invention. You may.

1: Information processing device 16: Display unit 18: Bus 31: Storage unit 32: Acquisition unit 33, 33a, 33b: Judgment unit 41: Setting unit 42: Extraction unit 43: Combined image generation unit 44: Specific unit 45: Similar image Specific part

Claims (17)

An information processing device used in the imaging process of photographing the inspection target in the deformation analysis for detecting the deformation of the inspection target by analyzing the combined image obtained by combining a plurality of images of the inspection target. ,
An acquisition means for acquiring a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so as to generate a superposed region in the photographing range with respect to the inspection target, and an acquisition means for acquiring a plurality of images as an image group.
An information processing device including a determination means for determining whether or not an image defect region is generated in a combined image generated from the image group by using an image included in the image group acquired by the acquisition means. The information processing device according to claim 1, wherein the determination means determines whether or not the image defect region occurs for each sub-image group corresponding to the sub-region that is a part of the inspection target. The determination means
A setting means for setting a plurality of processing target areas in a direction intersecting the overlapping direction in the sub-region, and
It is provided with an extraction means for extracting a plurality of images corresponding to the processing target area.
The information processing apparatus according to claim 2, wherein it is determined whether or not the image defect region is generated for each processing target region. The determination means performs preprocessing to eliminate defective images satisfying a predetermined condition from the images constituting the image group.
The information processing apparatus according to claim 3, wherein the setting means sets the processing target area so as to include a position corresponding to a defect image excluded in the preprocessing. The determination means
An image combining means for generating a combined image from the sub-image group is provided.
The information processing apparatus according to any one of claims 2 to 4, wherein the presence or absence of the image defect region is determined based on the combined image generated by the image combining means. The determination means
A specific means for extracting two adjacent images from the sub-image group as a set image and specifying feature points corresponding to each other in the set image as corresponding points is provided.
The information processing apparatus according to any one of claims 2 to 4, wherein it is determined whether or not the image defect region is generated depending on whether or not the number of the specified corresponding points is equal to or less than a predetermined threshold value. The determination means
A similar image detecting means for identifying a target image from the sub-image group and detecting a similar image similar to the specified target image from the sub-image group is provided.
The information processing apparatus according to any one of claims 2 to 4, wherein it is determined whether or not the image defect region is generated depending on whether or not a similar image is detected. The information processing apparatus according to any one of claims 1 to 7, further comprising a display means for displaying a determination result by the determination means. A data amount reducing means for performing a process of reducing the data amount of the image included in the image group acquired by the acquisition means is provided.
The information processing apparatus according to any one of claims 1 to 8, wherein the determination means determines whether or not the image defect region is generated by using the image of the image group in which the amount of data is reduced. Deformation analysis that detects deformation of the inspection target by executing the first image combination processing that combines a plurality of images of the inspection target and analyzing the combined image generated by the first image combination processing. In the information processing device used in the imaging process of photographing the inspection target,
An acquisition means for acquiring a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so as to generate a superposed region in the photographing range with respect to the inspection target, and an acquisition means for acquiring a plurality of images as an image group.
A combined image generating means that executes a second image combining process with a processing amount smaller than that of the first image combining process and generates a combined image from the images included in the image group acquired by the acquiring means.
An information processing device including a display means for displaying the generated combined image. The information processing according to claim 10, wherein the combined image generation means extracts a plurality of images corresponding to a part of a region in the inspection target from the image group, and generates a combined image using the extracted plurality of images. apparatus. A setting means for setting a plurality of processing target areas in a direction intersecting the overlapping direction in the inspection target, and
It is provided with an extraction means for extracting a plurality of images corresponding to each of the processing target areas.
The information processing apparatus according to claim 11, wherein the combined image generation means generates a combined image for each processing target region from a plurality of images extracted by the extraction means. A data amount reducing means for performing a process of reducing the data amount of the image included in the image group acquired by the acquisition means is provided.
The information processing apparatus according to any one of claims 10 to 12, wherein the combined image generation means generates a combined image using an image of the image group in which the amount of data is reduced. It is a shooting support method executed in the shooting process of shooting the inspection target in the deformation analysis for detecting the deformation of the inspection target by analyzing the combined image obtained by combining a plurality of images of the inspection target. hand,
A step of acquiring a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so that a superposed region is generated in the imaging range with respect to the inspection target, and a step of acquiring a plurality of images as an image group.
A photographing support method in which a computer executes a step of determining whether or not an image defect region occurs in a combined image generated from the image group using an image included in the image group. Deformation analysis that detects the deformation of the inspection target by executing the first image combination processing that combines a plurality of images of the inspection target and analyzing the combined image generated by the first image combination processing. Is a shooting support method executed in the shooting process of shooting the inspection target.
A step of acquiring a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so that a superposed region is generated in the imaging range with respect to the inspection target, and a step of acquiring a plurality of images as an image group.
A step of executing a second image combining process in which the amount of processing is smaller than that of the first image combining process and generating a combined image from the images included in the image group.
A shooting support method in which a computer executes a step of displaying the generated combined image. It is a shooting support program that is executed in the shooting process of shooting the inspection target in the deformation analysis that detects the deformation of the inspection target by analyzing the combined image that combines multiple images of the inspection target. hand,
A process of acquiring a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so that a superposed region is generated in the imaging range with respect to the inspection target, and a process of acquiring a plurality of images as an image group.
A shooting support program for causing a computer to execute a process of determining whether or not an image defect region is generated in a combined image generated from the image group by using an image included in the image group. Deformation analysis that detects deformation of the inspection target by executing the first image combination processing that combines a plurality of images of the inspection target and analyzing the combined image generated by the first image combination processing. Is a shooting support program executed in the shooting process of shooting the inspection target.
A process of acquiring a plurality of images acquired by moving the imaging means two-dimensionally or three-dimensionally so that a superposed region is generated in the imaging range with respect to the inspection target, and a process of acquiring a plurality of images as an image group.
A process of executing a second image combination process in which the amount of processing is smaller than that of the first image combination process and generating a combined image from the images included in the image group.
A shooting support program for causing a computer to execute a process of displaying the generated combined image.

Images