JP2018169967A - Image synthesis program, method, and device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image synthesis program, method, and device capable of obtaining an image of a subject from a plurality of images obtained by photographing the subject on an individual portion basis even in a case where the plurality of images includes poor image features.SOLUTION: Based on each of respective image features of two images which are included in a plurality of images obtained by photographing an object on an individual portion basis and which are sequentially photographed, a position relationship specification trial part 16 specifies positional relationship between portions of the objects photographed in the two images. Moreover, a camera work estimation section 18 estimates a camera work based on the specified positional relationship. Moreover, based on the estimated camera work, the positional relationship determination part determines a position of the image that could not be specified. Then an image synthesis part 26 synthesizes a plurality of images.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image composition program, an image composition method, and an image composition apparatus.

  Acquire a plurality of frame images that are continuous in time series from a moving image or a plurality of still images arranged in a time series, and generate at least two frame images of a plurality of frame images to generate one still image. A first technique has been proposed.

  Also, camera shake correction is performed by estimating the camerawork trajectory intended by the user from the trajectory of the imaging result detected from the motion between the fields of the imaging result or between the frames, and correcting the motion between the fields or between the frames. A second technique for performing the above has also been proposed.

  In addition, a third technique for classifying the content of video shot by a video camera or the like according to camera work at the time of shooting has been proposed. In the third technique, a rough moving direction of a video pattern is detected from a moving image, and camera work is classified into one of three types of still, pan, and zoom based on the detected rough moving direction.

Japanese Patent Laid-Open No. 2005-210428 JP 2008-005109 A JP-A-10-243340

  For a structure such as a tunnel or a bridge, for example, an inspection is periodically performed to check whether damage such as a crack has occurred on the wall surface of the tunnel or the floor plate of the bridge. In this work, usually the presence or absence of damage is visually inspected, but the object is photographed from a position distant from the visual inspection, and the inspection can be performed by referring to the image obtained by the photographing. It is expected that labor saving will be realized.

  However, if it is assumed that the object is photographed with a resolution that can confirm whether there is damage or not, the size of the object is very large compared to the size of the damage. Each shot is taken in a plurality of images. As an example, the size of the object reaches several to several tens [m] for a crack width of 0.1 [mm]. Then, in order to grasp where the damage found on the image is located in the entire object, a plurality of images taken for each part of the object are combined into one image having the entire object as a subject. Is required.

  Regarding the synthesis of a plurality of images, the first technique calculates a relative position of an image by obtaining a translation vector by image pattern matching or feature point tracking. However, the plurality of images obtained by capturing the object for each part include an image in which the background area does not exist in the image and a part of the object occupies the entire surface. The image features used for matching and feature point tracking are poor. In the first technique, the accuracy of pattern matching and feature point tracking is remarkably reduced for images with poor image features, so the relative position with other images cannot be calculated, or the relative position calculation accuracy is greatly increased. descend.

  The second technique detects motion vectors in each part of the image by tracking feature points, detects a group of motion vectors belonging to the background, and averages the motion vectors belonging to the detected group, so that camera shake and camera The camera motion vector for the workpiece is calculated. However, the second technique does not function effectively because the detection accuracy of the motion vector is remarkably lowered in an image with poor image characteristics, and a group of motion vectors belonging to the background cannot be detected in an image having no background region.

  In the third technique, a rough moving direction of the luminance pattern is obtained based on the sign of the product of the spatial differential value and the temporal differential value of the image. However, a clear peak does not appear in the spatial differential value of the image in an image with poor image characteristics, and the peak of the temporal differential value is interrupted at a time corresponding to an image with poor image characteristics. Therefore, the third technique does not function effectively for an image with poor image characteristics.

  In one aspect, an object of the present invention is to obtain an image of an object from a plurality of images even when an image with poor image characteristics is included in the plurality of images obtained by capturing the object for each part.

  In one embodiment, based on the image features of the processed image and the comparative image in which the processing order is continuous with the processed image among a plurality of images in which the target is captured for each part, the target image captured in the comparative image The positional relationship of the part of the object shown in the processed image with respect to the part is specified. Further, camera work is estimated from the positional relationship of the identified processed image. Then, based on the estimated camera work, the position of the part of the object shown in the image for which the positional relationship could not be specified is determined, and a plurality of images are combined.

  As one aspect, there is an effect that an image of an object can be obtained from the plurality of images even when an image with poor image characteristics is included in the plurality of images obtained by photographing the object for each part.

It is a functional block diagram of an image composition device. It is an image figure which shows an example of imaging | photography of the target object by an operator. It is an image figure which shows an example of the several image at the time of image | photographing a target object for every part, and the imaging range of each image. It is an image figure which shows an example of the pattern of camera work. It is a chart which shows an example of camera work pattern information. It is a schematic block diagram of a computer that functions as an image composition device. It is a flowchart which shows an example of an image composition process. It is an image figure which shows the imaging range of each image at the time of image | photographing a target object for every part, and the imaging | photography order of each image in time series. It is a flowchart which shows an example of an image relative position estimation process. It is a flowchart which shows an example of the image relative position estimation process 2. FIG. It is an image figure which shows an example of the search range in the matching of an image. It is an image figure which shows an example of the calculation of the relative coordinate and approximate direction in the matching of an image. It is an image figure explaining the process which matches the image of imaging | photography order = 2 with the image of imaging | photography order = 1. It is an image figure explaining the process which matches the image of imaging | photography order = 3 with the image of imaging | photography order = 2. It is an image figure explaining the process which matches the image of imaging | photography order = 4 with the image of imaging | photography order = 3. It is an image figure explaining the process which matches the image of imaging | photography order = 5 with the image of imaging | photography order = 4. It is an image figure explaining the process which matches the image of imaging | photography order = 5 with the image of imaging | photography order = 1. It is an image figure explaining the process which matches the image of imaging | photography order = 6 with the image of imaging | photography order = 5. It is a flowchart which shows an example of a synthesized image generation process. It is an image figure explaining the process which arrange | positions only the image in which positional relationship with the other image was specified, and produces a whole image. It is an image figure explaining the process which determines the position of the image of imaging | photography order = 6. It is an image figure which shows an example of the synthesized image finally obtained. It is an image figure which shows an example of a 1st output image. It is an image figure which shows an example of a 2nd output image. It is an image figure which shows an example of the several image at the time of image | photographing a target object for every part, and the imaging range of each image.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. An image composition device 10 shown in FIG. 1 includes an image acquisition unit 12, a crack detection unit 14, a positional relationship specifying trial unit 16, a camera work estimation unit 18, a storage unit 20 that stores camera work pattern information 22, and a positional relationship determination unit 24. The image composition unit 26 and the image output unit 28 are included.

  In the present embodiment, as shown in FIG. 2, an operator 32 operates an imaging device 34 on an inspection object 30 (hereinafter simply referred to as “object 30”) such as a tunnel wall surface or a bridge floor board. A plurality of images are taken for each portion of the object 30. In FIG. 2, a crack “30A” is given to the crack generated in the object 30. As shown by a broken line in FIG. 3, the worker 32 performs the shooting operation of the object 30 each time so that adjacent shooting ranges partially overlap each other. When the shooting range indicated by the broken line in FIG. 3 is shot in each shooting of the object 30, images I1 to I12 shown in FIG. 3 are obtained.

  Also, the images I1 to I12 shown in FIG. 3 are obtained when the operator 32 takes a picture of the object 30 a plurality of times while performing camerawork for moving the shooting range of the shooting device 34 along the pattern 1 shown in FIG. It is obtained with. However, the camera work when the operator 32 captures the object 30 is not limited to the pattern 1, and as the camera work pattern, for example, any one of the patterns 1 to 20 shown in FIG. 4 can be implemented. The camerawork pattern is not limited to the patterns 1 to 20 shown in FIG.

  As shown in FIG. 5 as an example, the camera work pattern information 22 stored in the storage unit 20 is a combination of moving directions of shooting ranges in which individual camera work patterns are switched in time series (the first direction in FIG. 5). (D1) to 4th direction (D4)). In the camerawork pattern information 22, different pattern numbers are assigned to information representing individual camerawork patterns.

  The image acquisition unit 12 of the image composition device 10 acquires a plurality of images (for example, images I1 to I12 shown in FIG. 3) for each portion of the captured object 30 from the imaging device 34. The crack detection unit 14 searches the plurality of images acquired by the image acquisition unit 12 for the presence or absence of image features corresponding to damage such as cracks (hereinafter simply referred to as “cracks”). When the crack is detected by the search, the crack detection unit 14 detects properties such as the width, length, and shape of the detected crack, and specifies information indicating the property of the crack to identify the image from which the crack has been detected. Output together with the information.

  The positional relationship specifying trial unit 16 sequentially extracts two images having a continuous shooting order from the plurality of images acquired by the image acquisition unit 12, and based on the image characteristics of the two extracted images, the two images Attempt to specify the positional relationship of the part of the object 30 that appears in The camera work estimating unit 18 is based on the positional relationship between the plurality of images whose positional relationship has been specified by the positional relationship specifying trial unit 16 and the camera work pattern information 22 stored in the storage unit 20. Estimates the camera work when the object 30 is photographed. The positional relationship determination unit 24 determines the position of an image whose positional relationship with another image is not specified by the positional relationship specifying trial unit 16 based on the camera work estimated by the camera work estimation unit 18.

  The image composition unit 26 uses the positional relationship between the plurality of images identified by the positional relationship identification trial unit 16 and the positions of the images determined by the positional relationship determination unit 24 to obtain a plurality of images acquired by the image acquisition unit 12. Are combined into one image having the entire object 30 as a subject. In addition, the image composition unit 26 divides the crack information output from the crack detection unit 14 into a synthesized image (first output image) and an image before synthesis (second output image) in which cracks are detected. Superimpose.

  The image output unit 28 outputs the first output image and the second output image generated by the image composition unit 26. The first output image and the second output image output by the image output unit 28 are visually confirmed by, for example, the operator 32 and the like and used for inspection of the object 30.

  In FIG. 1, the positional relationship specifying trial unit 16 is an example of a specifying unit in the present invention, the camera work estimating unit 18 is an example of an estimating unit in the present invention, and the positional relationship determining unit 24 and the image composition unit 26 are It is an example of the determination part in this invention.

  In the present embodiment, the image composition device 10 is realized by, for example, a computer 50 shown in FIG. The computer 50 includes a CPU 52, a memory 54, a non-volatile storage unit 56, an input unit 58, a display unit 60, a read / write device (R / W) 62 that reads and writes data to and from the recording medium 64, and a communication I / F part 66 is included. The CPU 52, the memory 54, the storage unit 56, the input unit 58, the display unit 60, the R / W 62, and the communication I / F unit 66 are connected to each other via a bus 68. The image composition device 10 can communicate with the photographing device 34 via a network connected to the communication I / F unit 66.

  The storage unit 56 is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, or the like. The storage unit 56 stores an image composition program 70 for causing the computer 50 to function as the image composition apparatus 10. The CPU 52 reads the image composition program 70 from the storage unit 56 and develops it in the memory 54, and sequentially executes each process included in the image composition program 70.

  The image composition program 70 includes an image acquisition process 72, a crack detection process 74, a position relationship identification trial process 76, a camera work estimation process 78, a position relationship determination process 80, an image composition process 82 and an image output process 84. The CPU 52 operates as the image acquisition unit 12 illustrated in FIG. 1 by executing the image acquisition process 72. Further, the CPU 52 operates as the crack detection unit 14 illustrated in FIG. 1 by executing the crack detection process 74. Further, the CPU 52 operates as the positional relationship specifying trial unit 16 shown in FIG. 1 by executing the positional relationship specifying trial process 76. Further, the CPU 52 operates as the camera work estimation unit 18 shown in FIG. 1 by executing the camera work estimation process 78. Further, the CPU 52 operates as the positional relationship determination unit 24 illustrated in FIG. 1 by executing the positional relationship determination process 80. The CPU 52 operates as the image composition unit 26 illustrated in FIG. 1 by executing the image composition process 82. The CPU 52 operates as the image output unit 28 shown in FIG. 1 by executing the image output process 84.

  As a result, the computer 50 that has executed the image composition program 70 functions as the image composition apparatus 10. The image composition program 70 is an example of an image composition program according to the disclosed technology. The storage unit 56 is provided with a camera work pattern information storage area 86, and the camera work pattern information storage area 86 stores camera work pattern information 22. Thereby, the memory | storage part 56 functions as the memory | storage part 20 shown in FIG.

  Note that the image composition device 10 can be realized by, for example, a semiconductor integrated circuit, more specifically, an ASIC (Application Specific Integrated Circuit) or the like.

  Next, as an operation of the present embodiment, referring to FIG. 7, after the operator 32 completes the operation of operating the imaging device 34 to capture a plurality of images of the object 30 for each part of the object 30, An image composition process executed by the image composition apparatus 10 will be described.

  In step 100 of the image composition processing, the image acquisition unit 12 acquires a plurality of images (for example, images I1 to I12 shown in FIG. 3) obtained by capturing the object 30 for each part from the imaging device 34. In step 102, the image acquisition unit 12 recognizes the shooting times of the plurality of images acquired in step 100, sorts the plurality of images in ascending order of the shooting times, and assigns the image numbers in ascending order of the shooting times to the individual images. Give. For example, if the image is in JPEG (Joint Photographic Experts Group) format, the image shooting time is set in EXIF (Exchangeable Image File Format) information. As a result of the sorting in step 102, as shown in an example in FIG. 8, the images I1 to I12 obtained by shooting the object 30 for each part are arranged in order of shooting time.

  In step 104, the crack detection unit 14 searches for a crack in each of the plurality of images acquired by the image acquisition unit 12. When the crack is detected by the search, the crack detection unit 14 detects properties such as the width, length, and shape of the detected crack, and specifies information indicating the property of the crack to identify the image from which the crack has been detected. Together with information to be stored in the memory 54 or the like.

  In the next step 106, the positional relationship specifying trial unit 16 and the camera work estimation unit 18 perform an image relative position estimation process. The details of the image relative position estimation process will be described below with reference to FIG.

  In step 120 of the image relative position estimation process, the positional relationship specifying trial unit 16 sets 2 to the image number cn indicating the processed image (the image to be processed), and indicates the comparison image (the image that matches the processed image). Set 1 to the image number rn. The positional relationship specifying trial unit 16 also identifies a variable i for identifying the order of the approximate moving direction of the shooting position (hereinafter referred to as “schematic direction”) and each row along the main shooting direction of the camera work. 1 is set for each variable j, and 0 is set for the number of images L (j) in the jth row.

  In the present embodiment, the main shooting direction of the camera work corresponds to the first direction (D1) in the camera work pattern information 22 shown in FIG. 5, and the second direction (D2) in the camera work pattern information 22 is the camera work. Is referred to as the sub-shooting direction.

  In the next step 122, the positional relationship specifying trial unit 16 matches the processed image with the image number cn (hereinafter referred to as “processed image cn”) with the comparison image with the image number rn (hereinafter referred to as “comparative image rn”). I do. This matching is performed, for example, by moving the processed image cn in the X and Y directions in increments of one pixel within the search range 90 that includes all the relative positions P1 to P4 (see FIG. 11) of the processed image cn with respect to the comparison image rn. The correlation evaluation value between the comparative image rn and the processed image cn is calculated at each relative position.

  The relative position P1 is a position where the lower right corner of the processed image cn overlaps the upper left corner of the comparative image rn by a predetermined width (for example, one pixel width), and the relative position P2 is the lower left corner of the processed image cn. It is a position that overlaps the upper right corner of rn by a predetermined width. The relative position P3 is a position where the upper right corner of the processed image cn overlaps with the lower left corner of the comparative image rn by a predetermined width, and the relative position P4 is the upper left corner of the processed image cn of the lower right corner of the comparative image rn and a predetermined width. It is an overlapping position. As the correlation evaluation value, any of various known parameters for evaluating the correlation between images can be used.

  Further, the positional relationship specifying trial unit 16 extracts the maximum value from the correlation evaluation value calculated for each relative position in the search range 90, and determines whether or not the extracted maximum value is equal to or greater than a predetermined threshold value. If the maximum correlation evaluation value is less than the threshold value, it can be determined that the processed image cn is not continuous with the comparison image rn. On the other hand, if the maximum correlation evaluation value is greater than or equal to the threshold value, the correlation evaluation value indicates the maximum value. It can be determined that the processed image cn is connected (continuous) to the comparative image rn at the relative position.

  Therefore, when the maximum correlation evaluation value is greater than or equal to the threshold value, the positional relationship specifying trial unit 16 uses the comparison image rn as the origin from the relative position where the correlation evaluation value indicates the maximum value, as shown in FIG. The relative coordinates (Xcn, Ycn) of the processed image cn are obtained. Further, the positional relationship specifying trial unit 16 obtains the angle of the vector 92 from the origin to the relative coordinates (Xcn, Ycn), and sets the approximate direction of the processed image cn relative to the comparison image rn to the right, up Categorized as either left or bottom. As an example, if the angle of the vector 92 is within the range of 315 ° to 45 °, the approximate direction is right, if it is within the range of 45 ° to 135 °, the approximate direction is up, and within the range of 135 ° to 225 °. If so, the approximate direction can be classified to the left, and the approximate direction can be classified to the upper if it is within the range of 225 ° to 315 °.

  When the matching between the processed image cn and the comparative image rn in step 122 is completed, in the next step 124, the positional relationship specifying trial unit 16 determines whether or not the processed image cn is continuous with the comparative image rn. If the processed image cn and the comparative image rn are images including the edges of the object 30 such as images I1 and I2 shown in FIG. 8, for example, the correlation evaluation value is calculated from the image features corresponding to the edges of the object 30. The maximum value of becomes greater than or equal to the threshold value.

  However, for example, when the shooting range of the comparative image rn is located on the left side of the shooting range of the image I1 shown in FIG. 8, the image characteristics corresponding to the edge of the object 30 does not exist in the comparative image rn. The maximum correlation evaluation value between cn and the comparative image rn is less than the threshold value. In this case, the determination at step 124 is negative and the routine proceeds to step 126. In step 126, the positional relationship specifying trial unit 16 registers the image number cn of the processed image in the positional unknown table for registering information of an image whose positional relationship with other images is unknown.

  In the next step 128, the positional relationship specifying trial unit 16 determines whether or not the image number cn has reached the total number of images acquired by the image acquisition unit 12. If the determination in step 128 is negative, the process proceeds to step 130. In step 130, the positional relationship specifying trial unit 16 increments the image numbers rn and cn by 1, and returns to step 122. As a result, while the determinations of steps 124 and 128 are respectively denied, steps 122 to 130 are repeated, and the maximum correlation evaluation value is greater than or equal to the threshold value while switching the processed image cn and the comparative image rn in ascending order of the photographing order. A set of the processed image cn and the comparison image rn is searched.

  Note that the determination in step 128 is affirmative because there is a set of a processed image cn and a comparative image rn in which the maximum correlation evaluation value is greater than or equal to a threshold value among the plurality of images acquired by the image acquisition unit 12. This is the case. In this case, it is conceivable that the photographing of the object 30 has failed, so that an error notification process or the like is performed and the image relative position estimation process is terminated.

  On the other hand, if the processed image cn and the comparative image rn are images including the edges of the object 30 such as the images I1 and I2 shown in FIG. The maximum evaluation value is equal to or greater than the threshold value. As a result, the determination at step 124 is affirmed and the routine proceeds to step 132.

  In step 132, the positional relationship specifying trial unit 16 sets the approximate direction of the processed image cn for the comparison image rn classified in step 122 as the approximate direction Di. As an example, when the image I1 shown in FIG. 8 is the comparative image rn and the image I2 is the processed image cn, as shown in FIG. 13, the approximate direction of the image I2 with respect to the image I1 is classified as “right”. “Right” is set in D1. The positional relationship specifying trial unit 16 stores the set approximate direction Di in the memory 54 and the like together with the relative coordinates (X2, Y2) of the image I2 with the image I1 as the origin. Further, the positional relationship specifying trial unit 16 sets the image number rn of the comparison image corresponding to the origin of the relative coordinates of the processed image cn to the reference image REFcn, increments the number of images L (j) by 1, and sets the reference image REFcn. The number of images L (j) is also stored in the memory 54 or the like. In the example shown in FIG. 13, the images I1 and I2 are images whose positional relationships are specified (reference images REFcn and relative coordinates (Xcn, Ycn) are stored in the memory 54, etc.), and the number of images after increment L (1) = 1.

  In the next step 134, the camera work estimation unit 18 extracts all patterns whose first direction matches the general direction Di from the individual camera work patterns defined in the camera work pattern information 22. For example, in the example shown in FIG. 13, since the approximate direction D1 is “right”, patterns whose first direction is “right”, that is, pattern numbers 1, 2, 5, 6, 7, and 10 are extracted. And the camerawork estimation part 18 sets the pattern number of all the extracted patterns to a pattern table.

  In step 136, the positional relationship specifying trial unit 16 increments the image numbers rn and cn by 1, respectively. In step 138, the positional relationship specifying trial unit 16 matches the processed image cn and the comparative image rn in the same manner as in step 122 described above. In step 140, the positional relationship specifying trial unit 16 determines whether or not the processed image cn is continuous with the comparative image rn, as in step 124 described above.

  If the processed image cn and the comparative image rn are images including the edges of the object 30 such as images I2 and I3 shown in FIG. 8, for example, the correlation evaluation value is calculated from the image features corresponding to the edges of the object 30. The maximum value of becomes greater than or equal to the threshold value. In this case, the determination at step 140 is affirmed and the routine proceeds to step 142. In step 142, the positional relationship specifying trial unit 16 determines whether or not the approximate direction of the processed image cn with respect to the comparative image rn matches the approximate direction Di.

  As an example, when the image I2 shown in FIG. 8 is the comparative image rn and the image I3 is the processed image cn, the schematic direction of the image I3 with respect to the image I2 is classified as “right” as shown in FIG. It matches D1. In such a case, the determination at step 142 is affirmed and the routine proceeds to step 144. In step 144, the positional relationship specifying trial unit 16 stores the relative coordinates (X3, Y3) of the image I3 with the image I2 as the origin in the memory 54 or the like. Further, the positional relationship specifying trial unit 16 sets the image number = 2 of the image I2 corresponding to the origin of the relative coordinates of the image I3 to the reference image REF3, increments the number of images L (1) by 1, and the reference image REF3. The number of images L (1) is also stored in the memory 54 or the like. In the example shown in FIG. 14, the image I3 is added as an image in which the positional relationship with another image is specified (an image in which the reference image REFcn and the relative coordinates (Xcn, Ycn) are stored in the memory 54 etc.). The number of images L (1) = 2.

  When the processing of step 144 is performed, the routine proceeds to step 154. In step 154, the positional relationship specifying trial unit 16 determines whether or not the image number cn has reached the total number of images acquired by the image acquisition unit 12. If the determination in step 154 is negative, the process returns to step 136. As a result, while the determinations at steps 140 and 142 are both affirmed and the determination at step 154 is negative, step 144 is repeated while incrementing the image numbers rn and cn by 1, respectively.

  For example, if the image I3 shown in FIG. 8 becomes the comparative image rn and the image I4 becomes the processed image cn with the increment of the image numbers rn and cn, as shown in FIG. 15, the outline of the image I4 with respect to the image I3. The direction is classified as “right”. In this example, an image I4 is added as an image whose positional relationship with other images is specified, and the number of images after increment L (1) = 3.

  By the way, when the worker 32 performs the camera work corresponding to the pattern 2 shown in FIG. 4 when shooting the object 30, for example, the images I1 to I4 shown in FIG. The image I8 shown is taken. As a result, a scene occurs in which the image I4 becomes the comparative image rn and the image I8 becomes the processed image cn. In this case, although not shown, the approximate direction of the processed image cn with respect to the comparative image rn is classified as “down” and does not coincide with “right” in the approximate direction D1, so the determination in step 142 is negative and the process proceeds to step 146. Transition.

  In step 146, the positional relationship specifying trial unit 16 increments the variables i and j by one, sets 1 to the number of images L (j), and sets the approximate direction of the processed image cn relative to the comparative image rn to the approximate direction Di. Set. The positional relationship specifying trial unit 16 then sets the set approximate direction Di, the relative coordinates (Xcn, Ycn) of the processed image cn with the comparison image rn as the origin, the reference image REFcn set with the image number rn of the comparison image, and The number of images L (j) is stored in the memory 54 or the like. For example, when the image I4 becomes the comparative image rn and the image I8 becomes the processed image cn, the approximate direction D2 is set to “down” and the number of images L (2) = 1.

  In the next step 150, the camera work estimation unit 18 determines whether or not the number of pattern numbers registered in the pattern table is one. If the determination in step 150 is negative, the process proceeds to step 152.

  In step 152, the camera work estimation unit 18 determines that the first direction to the i-th direction specified in the camera work pattern information 22 are the approximate directions D1 to Di for the individual patterns whose pattern numbers are registered in the pattern table. And whether or not all match. Then, the camera work estimation unit 18 deletes all the pattern numbers from the pattern table for the patterns in which the first direction to the i-th direction include a mismatch with the approximate directions D1 to Di.

  For example, when the approximate direction D1 is “right” and the approximate direction D2 is “down”, when the “right” of the approximate direction D1 is found, the patterns 1, 2, 5, 6 with the first direction being “right” , 7, 10 are registered in the pattern table. Thereafter, when “down” in the approximate direction D2 is found, the patterns 6, 7, and 10 are deleted from the pattern table because the second direction is not “down” and does not match the approximate direction D2.

  When the process of step 152 is performed, the process proceeds to step 154. If the determination in step 150 is affirmative, step 152 is skipped and the process proceeds to step 154. If the number of pattern numbers registered in the pattern table decreases to 1, step 152 is not executed because the determination in step 150 is affirmed. Thus, when the number of pattern numbers registered in the pattern table becomes 1, the camera work estimation unit 18 has completed the estimation of the camera work pattern.

  On the other hand, when the operator 32 takes a camera work corresponding to the pattern 1 shown in FIG. 4 when the object 30 is taken, for example, after the images I1 to I4 shown in FIG. By taking a picture, a scene occurs in which the image I4 becomes the comparative image rn and the image I5 becomes the processed image cn. In this case, as shown in FIG. 16 as an example, when the maximum correlation evaluation value between the comparison image rn and the processed image cn is less than the threshold, the determination in step 140 is denied and the process proceeds to step 148. In step 148, the positional relationship specifying trial unit 16 performs image relative position estimation processing 2. Hereinafter, the image relative position estimation process 2 will be described with reference to FIG.

In step 160 of the image relative position estimation process 2, the positional relationship specifying trial unit 16 determines whether or not the image cn-1 is registered in the position unknown table. For example, when I5 shown in FIG. 8 is a processed image, the image cn-1 is I4. I4 is not registered in the position unknown table because the relative position with respect to I3 is determined as shown in FIG. Therefore, the determination at step 160 is negative and the process proceeds to step 162. On the other hand, when the processed image is I7, the image cn-1 is I6. As shown in FIG. 18, I6 is registered in the position unknown table because the relative position with respect to I5 is not fixed. Therefore, the determination at step 160 is affirmed, and the routine proceeds to step 178.

In step 162, the positional relationship specifying trial unit 16 determines whether the number of images L (j) included in the approximate direction of the camera work being processed is one. When L (j) is not 1, a portion where the main shooting direction is switched to the sub shooting direction of the camera work with a fixed main shooting direction as in patterns 1, 3, 6, 8, 11, 13, 16, and 18 in FIG. As shown in FIG. When L (j) is 1, the direction of camera work in which the main shooting direction changes as in patterns 2, 4, 5, 7, 9, 10, 12, 14, 15, 17, 19, 20 in FIG. As a changed part, the process proceeds to step 170.

In step 164, the positional relationship specifying trial unit 16 performs matching similar to that in step 122 described above to obtain the relative position at the portion of the camera work in which the main shooting direction is constant and switches to the sub shooting direction. And the comparison image cn-L (j) -1. For example, when the processed image is I5 shown in FIG. 8, cn = 5, j = 1, L (1) = 3, and the comparison image is I1, and therefore I5 and I1 are matched.

In step 166, the positional relationship specifying trial unit 16 determines whether or not the processed image cn and the comparison image cn-L (j) -1 are continuous, as in step 124 described above. If the determination at step 166 is affirmative, the routine proceeds to step 168.

In step 168, the positional relationship specifying trial unit 16 increments i and j by 1, and sets the number of images L (j) to 1. Furthermore, “oblique” is added to the approximate direction Di with respect to the approximate direction with respect to the processed image cn and the comparison image cn-L (j) −1. For example, when the image I5 shown in FIG. 8 is a processed image and the image I1 is a comparative image, as shown in FIG. 17, the approximate direction of the image I5 with respect to the image I1 is classified as “down”, and the approximate direction D2 includes , “Below” with “bevel” added is set. The positional relationship specifying trial unit 16 stores the set approximate direction Di in the memory 54 and the like together with the relative coordinates (Xcn, Ycn) of the processed image with the comparison image as the origin. Further, the positional relationship specifying trial unit 16 sets the image number cn-L (j) -1 of the comparison image corresponding to the origin of the relative coordinates of the processed image cn as the reference image REFcn, and sets the reference image REFcn and the image number L ( j) is also stored in the memory 54 or the like.

In step 170, the camera work estimation unit 18 determines whether the number of pattern numbers registered in the pattern table is 1 as in step 150 described above. If the determination in step 170 is negative, the process proceeds to step 172.

In step 172, the camera work estimation unit 18 determines that the first direction to the i-th direction specified in the camera work pattern information 22 for each pattern registered in the pattern table is the same as in step 152 described above. It is determined whether or not all the approximate directions D1 to Di coincide. Then, the camera work estimation unit 18 deletes all the pattern numbers from the pattern table for the patterns in which the first direction to the i-th direction include a mismatch with the approximate directions D1 to Di. Further, when there is not one pattern registered in the pattern table, the pattern number having no priority identifier assigned to each pattern shown in FIG. 5 is deleted from the pattern table. For example, as shown in FIG. 25, it is assumed that images I1 'to I12' are taken with the camerawork pattern 2 of FIG. In this case, when the processing from the image I1 ′ to the image I5 ′ is completed and the image I6 ′ becomes a processed image, the pattern 2 and the pattern 5 are registered in the pattern table before step 172 is executed. Yes. In step 172, since the priority identifier of the pattern 2 shown in FIG. 5 is present and the priority identifier of the pattern 5 is not present, the camera work estimation unit 18 deletes the pattern 5 from the pattern table.

In step 174, the positional relationship specifying trial unit 16 increases i and j by 1, and sets the number of images L (j) to 1. Further, the i-th direction of the camerawork pattern remaining in the pattern table is read from the table of FIG. 5 and set to the approximate direction Di.

In step 176, as in step 126 described above, the positional relationship specifying trial unit 16 sets the image number cn of the processed image in the positional unknown table for registering information of the image whose positional relationship with other images is unknown. sign up.

In step 178, the positional relationship specifying trial unit 16 determines whether the main shooting direction of the pattern registered in the pattern table is constant, so that the registered pattern is 1, 3, 6, 8, 11 , 13, 16, or 18 is determined. If the determination in step 178 is affirmed, the process proceeds to step 180. If the determination in step 178 is negative, the process proceeds to step 182.

In step 180, the positional relationship specifying trial unit 16 performs matching similar to that in step 122 described above for the processed image cn and the comparison image cn-L (j) -L (j-1) -1. For example, if the processed image is I8 shown in FIG. 8, cn = 8, j = 3, L (3) = 2, L (2) = 1, and the comparison image is I4. Do.

In step 182, the positional relationship specifying trial unit 16 performs matching similar to that in step 122 described above, and performs processing image cn and comparison image cn-L (j) -L (j-1) -L (j-2) -1 Do about. For example, when the processed image is I8 ′ shown in FIG. 25, cn = 8, j = 3, L (3) = 2, L (2) = 1, L (1) = 3, and the comparison image is I1 ′. Therefore, I8 ′ and I1 ′ are matched.

In step 184, the positional relationship specifying trial unit 16 determines whether or not the processed image and the comparison image are continuous, as in step 124 described above. If the determination in step 184 is affirmed, the process proceeds to step 186. If the determination in step 184 is negative, the process proceeds to step 188.

In step 186, the positional relationship specifying trial unit 16 increases the number of images L (j) by one. Further, the relative coordinates (Xcn, Ycn) of the processed image with the comparison image as the origin are stored in the memory 54 or the like. Further, the image number of the comparison image corresponding to the origin of the relative coordinates of the processed image is set as the reference image REFcn, and the reference image REFcn and the image number L (j) are also stored in the memory 54 or the like. In step 188, the positional relationship specifying trial unit 16 increases the number of images L (j) by 1 and stores it in the memory 54 or the like. In step 190, as in step 126 described above, the positional relationship specifying trial unit 16 sets the image number cn of the processed image in the positional unknown table for registering information of an image whose positional relationship with other images is unknown. sign up.

  When the above-described image relative position estimation process 2 ends, the process proceeds to step 150 of the image relative position estimation process (FIG. 9). Accordingly, in the image relative position estimation process (FIG. 9), step 144 or step 146 or the above-described image relative position estimation process 2 is selectively executed for the images after image number = 2. As a result, the plurality of captured images include an image in which the positional relationship with other images can be specified, that is, an image in which the reference image REFcn and relative coordinates (Xcn, Ycn) are stored in the memory 54 and the like, And the image registered in the position unknown table cannot be identified.

  When the image relative position estimation process ends, the process proceeds to step 108 of the image composition process (FIG. 7). In step 108, the positional relationship determination unit 24 and the image composition unit 26 perform a composite image generation process. Details of the composite image generation processing will be described with reference to FIG.

  In step 200 of the composite image generation process, the image composition unit 26 sets 1 to the image number cn of the processed image. In the next step 202, the image composition unit 26 determines whether or not the image number cn of the processed image is registered in the position unknown table. If the determination in step 202 is affirmed, the process proceeds to step 204.

  In step 204, the image composition unit 26 increments the image number cn by 1 and returns to step 202. Thus, steps 202 and 204 are repeated until the determination at step 202 is negative. Therefore, when the image with the image number = 1 is registered in the position unknown table, the image number cn is incremented by 1 until an image not registered in the position unknown table appears.

  If an image that is not registered in the position unknown table appears, the determination in step 202 is negative and the process proceeds to step 206. At the time when the composite image generation process is performed, the number of pattern numbers registered in the pattern table is 1, and the camerawork estimation unit 18 has completed the estimation of the camerawork pattern. Therefore, in step 206, the image composition unit 26 expands the processed image data to a storage position corresponding to the estimated start point of the camera work pattern in the composite image storage area secured in advance in the memory 54. Arrange. For example, if the estimated camerawork pattern is pattern 1, the processed image data is developed and arranged at a position corresponding to the upper left corner that is the starting point of pattern 1.

  In the next step 208, the image composition unit 26 increments the image number cn by 1. In step 210, the image composition unit 26 determines whether or not the image number cn of the processed image is registered in the position unknown table. . If the determination in step 210 is affirmed, the process returns to step 208, and steps 208 and 210 are repeated until the determination in step 210 is affirmed. Thus, the image number cn is incremented by 1 until an image not registered in the position unknown table appears.

  If an image that is not registered in the position unknown table appears, the determination in step 210 is denied and the process proceeds to step 212. In step 212, the image composition unit 26 refers to the reference image REFcn and relative coordinates (Xcn, Ycn) corresponding to the processed image cn stored in the memory 54. In addition, the image composition unit 26 stores the XY coordinates of the position corresponding to the relative coordinates (Xcn, Ycn) with the reference image REFcn already developed and arranged in the composite image storage area as the origin, and the composite image storage of the reference image REFcn. Calculation is performed based on the XY coordinates in the region. Then, the image composition unit 26 expands and arranges the data of the processed image cn at a position corresponding to the calculated XY coordinates in the composite image storage area.

  In the next step 214, the image composition unit 26 determines whether or not the image number cn has reached the total number of images acquired by the image acquisition unit 12. If the determination in step 214 is negative, the process returns to step 208, and steps 208 to 214 are repeated until the determination in step 214 is positive. Accordingly, as shown in FIG. 20 as an example, among the images I1 to I12, the images except the images I6 and I7 whose image features are poor and whose image numbers are registered in the position unknown table are the reference image REFcn and the relative coordinates. Based on (Xcn, Ycn), each is developed and arranged in the composite image storage area.

  When the image number cn reaches the total number of images, the determination in step 214 is affirmed and the process proceeds to step 216. In step 216, the positional relationship determination unit 24 determines whether or not the position unknown table is empty. If the determination in step 216 is negative, the process proceeds to step 218. In step 218, the positional relationship determination unit 24 selects and selects the smallest image number cn from the image numbers registered in the position unknown table. The taken image number cn is taken out from the position unknown table.

  In the next step 220, the positional relationship determination unit 24 determines the upper and lower sides adjacent to the processed image cn based on the camera pattern information 22 of the only pattern number registered in the pattern table and the arrangement of the number of images L (j). Recognize the image numbers of the left and right four-direction images. Each element of the array of the number of images L (j) has the value of the number of images minus 1 for each group of images captured until the approximate direction of movement of the imaging range changes in the camera work performed by the operator 32. Is set. Further, the approximate direction of movement of the shooting range when images of individual image groups are shot is defined in the camera work pattern information 22. Therefore, from these pieces of information, if the camerawork pattern is pattern 1, for example, it can be recognized that the images in the four directions of the top, bottom, left and right adjacent to the image I6 shown in FIG. 20 are the images I2, I5, I7 and I10.

  In step 222, the positional relationship determination unit 24 extracts an image that has been placed in the composite image storage area from the four images in the upper, lower, left, and right directions adjacent to the processed image cn whose image number has been recognized in step 220. For example, with respect to the image I6 shown in FIG. 20, the images I2, I5, I10 are arranged as images already arranged in the composite image storage area from the four images I2, I5, I7, I10 in the four directions up, down, left and right adjacent to the image I6. Is extracted.

  In the next step 224, the positional relationship determination unit 24 determines the processing image cn based on the approximate direction of the processed image cn with respect to the arranged image extracted in step 222 and the position of the image in the arranged composite image storage area. A position (XY coordinate) in the composite image storage area is determined. The determination of the position of the processed image can be realized by the following processing, for example.

  That is, first, the image extracted in step 222 that is adjacent to the processed image cn and already arranged in the composite image storage area is adjacent to the processed image cn in the X direction (the approximate direction is “left” or “right”). ) Images and images adjacent in the Y direction (the approximate direction is “up” or “down”). Next, the Y coordinate value in the composite image storage area of the processed image cn is determined from the Y coordinate value in the composite image storage area of the image adjacent to the processed image cn in the X direction. For example, if the number of arranged images adjacent to the processed image cn in the X direction is 1, the Y coordinate value of the image is set to the Y coordinate value in the composite image storage area of the processed image cn. May be. For example, if the number of arranged images adjacent to the processed image cn in the X direction is two, the average value of the Y coordinate values of the two images is stored in the composite image storage area of the processed image cn. You may set to the Y coordinate value in.

  Further, the X coordinate value in the composite image storage area of the processed image cn is determined from the X coordinate value in the composite image storage area of the image adjacent to the processed image cn in the Y direction. For example, if the number of arranged images adjacent to the processed image cn in the Y direction is 1, the X coordinate value of the image is set to the X coordinate value in the composite image storage area of the processed image cn. May be. For example, if the number of arranged images adjacent to the processed image cn in the Y direction is two, the average value of the X coordinate values of the two images is stored in the composite image storage area of the processed image cn. You may set to the X coordinate value in.

  FIG. 21 shows an example in which the position of the image I6 shown in FIG. 20 is determined from the coordinate values of the images I2 and I5. In this way, if there are at least one image adjacent to the processed image cn in the X direction and one image adjacent in the Y direction, the position of the processed image cn can be determined, so that one for each of the X and Y directions. The position of the processed image cn may be determined from the individual images.

  In the next step 226, the image composition unit 26 develops and arranges the data of the processed image cn at the position in the composite image storage area of the processed image cn determined in step 224. If step 226 is performed, it will return to step 216 and will repeat step 216-step 226 until determination of step 216 is affirmed. As a result, the images whose image numbers are registered in the position unknown table are developed and arranged in the composite image storage area in ascending order of the image numbers. Then, at the time when the position unknown table becomes empty and the determination in step 216 is affirmed, as shown in FIG. 22 as an example, a composite image in which all images acquired by the image acquisition unit 12 are combined is generated. The The composite image is an image having the entire object 30 as a subject.

  If the determination in step 216 is affirmative, the composite image generation process is terminated, and the process proceeds to step 110 of the image composition process (FIG. 7). In step 110, the image composition unit 26 reads out information related to the crack detected by the crack detection unit 14 from the memory 54 or the like. Then, the image composition unit 26 generates a first output image in which the crack is highlighted on the composite image from the composite image generated by the composite image generation process (FIG. 19) based on the acquired information on the crack. An example of the first output image generated by the image composition unit 26 is shown in FIG.

  In the next step 112, the image composition unit 26 selects an image in which the crack is detected by the crack detection unit 14 from the plurality of images acquired by the image acquisition unit 12 based on the read information on the crack. Then, the image composition unit 26 highlights the crack on the selected image (the image from which the crack has been detected) based on the information on the crack, and outputs the second output image to which the detailed information of the crack is added. Generate. An example of the second output image generated by the image composition unit 26 is shown in FIG.

  In step 114, the image output unit 28 outputs the first output image and the second output image generated by the image composition unit 26 by, for example, displaying them on the display unit 60, and the image composition process ends.

  By referring to the first output image output by the image output unit 28, the operator can easily grasp the number of cracks, the approximate position, and the like in the entire object 30. Further, the operator can easily grasp the properties of individual cracks generated on the object 30 by referring to the second output image output by the image output unit 28.

  As described above, in the present embodiment, the positional relationship specifying trial unit 16 performs the following processing using each of the plurality of images obtained by capturing the object 30 for each portion as a processed image. In other words, the positional relationship specifying trial unit 16 determines the object 30 that appears in the processed image for the portion of the object 30 that appears in the comparison image based on the image features of the processed image and the comparison image in which the processed image and the imaging order are continuous. Attempt to identify the positional relationship of the parts. Further, the camera work estimation unit 18 estimates camera work from the positional relationship of the images whose positional relationship has been identified by the positional relationship identification trial unit 16. The positional relationship determination unit 24 determines the position of the portion of the object 30 that appears in the image whose positional relationship could not be specified by the positional relationship specifying trial unit 16 based on the camera work estimated by the camera work estimation unit 18. To do. Then, the image composition unit 26 composes a plurality of images. Thereby, even when an image with poor image characteristics is included in a plurality of images taken for each part of the object, an image of the object can be obtained from the plurality of images.

  Moreover, in this embodiment, the positional relationship specification trial part 16 performs the following processes about the process image in which the said positional relationship with respect to the comparison image was not able to be specified. That is, the positional relationship specifying trial unit 16 has a positional relationship in which the portion of the object 30 that appears in the image is adjacent to the sub-shooting direction that intersects the main shooting direction of the camera work with respect to the portion of the object 30 that appears in the processed image. The possible image is selected from a plurality of images. Then, the positional relationship specifying trial unit 16 tries to specify the positional relationship of the processed image with respect to the selected image. As a result, even if the image characteristics of one of the processed image and the comparison image in which the shooting order is continuous are poor, or the shooting range on the object of the two images is separated, the position based on the image features The number of images for which the relationship is specified can be increased, and the accuracy of image synthesis is improved.

  In this embodiment, the camera work estimation unit 18 selects a pattern that does not match the positional relationship that the positional relationship specifying trial unit 16 can identify from among a plurality of camera work patterns stored in advance in the storage unit 20. Camerawork is estimated by excluding from the above. Thereby, compared with the case where the information showing a pattern is produced | generated from the positional relationship which the positional relationship specific trial part 16 was able to identify, the process which estimates a camera work can be simplified.

  In the above description, a mode has been described in which two images having a continuous shooting order are selected from a plurality of images acquired by the image acquisition unit 12 in ascending order of the shooting order. However, the present invention is not limited to this. May be selected in descending order of the photographing order.

  Moreover, although the camera work estimation part 18 estimated the camera work estimation in parallel with the specification of the positional relation of the image by the positional relation specification trial part 16 above, it is not limited to this. For example, after the positional relationship specifying trial unit 16 completes specifying the positional relationship of the images, the camera work estimating unit 18 determines the positional relationship of all the images for which the positional relationship specifying trial unit 16 has been able to specify the positional relationship. Camera work estimation may be performed.

  Further, in the above description, the general direction is the four directions of up, down, left, and right, and the mode in which the camerawork pattern is represented in these four directions has been described, but the present invention is not limited to this. For example, the second direction of the pattern 1 shown in FIG. 4 is “down” according to FIG. 5, but is exactly “the beginning of the next line (left diagonally down)”, and the second direction of the pattern 6 is According to FIG. 5, it is “up”, but it is precisely “the front of the previous line (upward left)”. In this way, the general direction may be further subdivided. In this case, for example, the second direction is different between the pattern 1 and the pattern 2, and the estimation of the camera work can be completed at an earlier stage.

  In the above description, the mode in which the image composition program 70 which is an example of the image composition program according to the present invention is stored (installed) in the storage unit 56 in advance has been described. However, the present invention is not limited to this, and the image composition program according to the present invention can be provided in a form recorded on a recording medium such as a CD-ROM, a DVD-ROM, or a memory card.

  All documents, patent applications and technical standards mentioned in this specification are to the same extent as if each individual document, patent application and technical standard were specifically and individually stated to be incorporated by reference. Incorporated by reference in the book.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
The processing for the portion of the object that appears in the comparison image based on the image characteristics of the processed image and the comparison image in which the imaging order is continuous with the processed image among a plurality of images in which the object is captured for each portion. Identify the positional relationship of the part of the object in the image,
Camera work is estimated from the positional relationship of the identified processed image,
An image composition program for causing a computer to execute a process of determining a position of a portion of the object in an image for which the positional relationship cannot be specified based on the estimated camera work and combining the plurality of images.

(Appendix 2)
For the processed image in which the positional relationship with respect to the comparison image cannot be specified, the portion of the object that appears in the image intersects the main shooting direction of the camera work with respect to the portion of the object that appears in the processed image. The image composition program according to appendix 1, wherein the image that may be in a positional relationship adjacent to a shooting direction is selected from the plurality of images, and the positional relationship of the processed image with respect to the selected image is tried.

(Appendix 3)
The image composition program according to appendix 1 or appendix 2, wherein the camerawork is estimated by excluding a pattern that does not match the positional relationship that can be identified from a plurality of camerawork patterns stored in advance in a storage unit .

(Appendix 4)
The processing for the portion of the object that appears in the comparison image based on the image characteristics of the processed image and the comparison image in which the imaging order is continuous with the processed image among a plurality of images in which the object is captured for each portion. Identify the positional relationship of the part of the object in the image,
Camera work is estimated from the positional relationship of the identified processed image,
An image composition method in which a computer executes a process of determining a position of a portion of the object in an image for which the positional relationship could not be specified based on the estimated camera work and combining the plurality of images.

(Appendix 5)
For the processed image in which the positional relationship with respect to the comparison image cannot be specified, the portion of the object that appears in the image intersects the main shooting direction of the camera work with respect to the portion of the object that appears in the processed image. The image composition method according to appendix 4, wherein the image that may be in a positional relationship adjacent to the imaging direction is selected from the plurality of images, and the positional relationship of the processed image with respect to the selected image is tried.

(Appendix 6)
The image synthesizing method according to appendix 4 or appendix 5, wherein the camerawork is estimated by excluding, from candidates, a pattern that does not match the identified positional relationship among a plurality of camerawork patterns stored in advance in a storage unit .

(Appendix 7)
The processing for the portion of the object that appears in the comparison image based on the image characteristics of the processed image and the comparison image in which the imaging order is continuous with the processed image among a plurality of images in which the object is captured for each portion. A specifying unit for specifying a positional relationship of the part of the object in the image;
An estimation unit that estimates camera work from the positional relationship of the identified processed image;
Based on the estimated camera work, a determination unit that determines the position of the part of the object in the image for which the positional relationship could not be specified, and combines the plurality of images;
An image synthesizing apparatus.

(Appendix 8)
For the processed image in which the positional relationship with respect to the comparison image could not be specified, the specifying unit is configured such that the portion of the object that appears in the image is the main image of the camera work. The supplementary note 7 wherein the image that may be in a positional relationship adjacent to the sub-photographing direction intersecting the direction is selected from the plurality of images, and the positional relationship of the processed image with respect to the selected image is tried. Image composition device.

(Appendix 9)
The estimation unit estimates the camera work by excluding, from candidates, a pattern that does not match the identified positional relationship among a plurality of camera work patterns stored in advance in the storage unit. The image composition apparatus described.

DESCRIPTION OF SYMBOLS 10 Image composition apparatus 16 Positional relationship specific trial part 18 Camerawork estimation part 20 Storage part 24 Positional relationship determination part 26 Image composition part 30 Object 32 Operator 34 Imaging device 50 Computer 52 CPU
54 Memory 56 Storage 70 Image composition program

Claims (5)

The processing for the portion of the object that appears in the comparison image based on the image characteristics of the processed image and the comparison image in which the imaging order is continuous with the processed image among a plurality of images in which the object is captured for each portion. Identify the positional relationship of the part of the object in the image,
Camera work is estimated from the positional relationship of the identified processed image,
An image composition program for causing a computer to execute a process of determining a position of a portion of the object in an image for which the positional relationship cannot be specified based on the estimated camera work and combining the plurality of images.   For the processed image in which the positional relationship with respect to the comparison image cannot be specified, the portion of the object that appears in the image intersects the main shooting direction of the camera work with respect to the portion of the object that appears in the processed image. The image synthesis program according to claim 1, wherein the image that may be in a positional relationship adjacent to a photographing direction is selected from the plurality of images, and the positional relationship of the processed image with respect to the selected image is tried.   3. The image according to claim 1, wherein the camera work is estimated by excluding, from candidates, a pattern that does not match the identified positional relationship among a plurality of camera work patterns stored in advance in a storage unit. 4. Synthesis program. The processing for the portion of the object that appears in the comparison image based on the image characteristics of the processed image and the comparison image in which the imaging order is continuous with the processed image among a plurality of images in which the object is captured for each portion. Identify the positional relationship of the part of the object in the image,
Camera work is estimated from the positional relationship of the identified processed image,
An image composition method in which a computer executes a process of determining a position of a portion of the object in an image for which the positional relationship could not be specified based on the estimated camera work and combining the plurality of images. The processing for the portion of the object that appears in the comparison image based on the image characteristics of the processed image and the comparison image in which the imaging order is continuous with the processed image among a plurality of images in which the object is captured for each portion. A specifying unit for specifying a positional relationship of the part of the object in the image;
An estimation unit that estimates camera work from the positional relationship of the identified processed image;
Based on the estimated camera work, a determination unit that determines the position of the part of the object in the image for which the positional relationship could not be specified, and combines the plurality of images;
An image synthesizing apparatus.