JP5925042B2 - Image synthesizing method, image synthesizing program, and image synthesizing apparatus - Google Patents

Description

  The present invention relates to an image composition method, an image composition program, and an image composition apparatus.

  2. Description of the Related Art Conventionally, a subject is divided into a plurality of images, and the captured partial images are combined to generate a combined image.

  By generating a composite image, an image having a wide viewing angle and high resolution can be obtained.

  Usually, two partial images photographed adjacent to each other have an overlapping area in which the same part of the subject is photographed. When generating a composite image, two partial images are arranged so that the respective overlapping areas coincide with each other, and a plurality of portions are arranged so that the positional relationship between adjacent partial images corresponds to the positional relationship at the time of shooting. Place each image.

  FIG. 1 is a diagram illustrating the synthesis of partial images according to a conventional example.

  In FIG. 1, a subject (not shown) is photographed by being divided into four partial images A1 to A4. Two partial images photographed adjacent to each other have an overlapping region in which the same part of the subject is photographed.

  The partial image A1 and the partial image A2 are arranged so that their overlapping regions V12 coincide. Here, a vector having the image gravity center G1 of the partial image A1 as a start point and the image gravity center G2 of the partial image A2 as an end point is referred to as a position vector P12.

  The overlapping area V12 is an area of the partial image A1 and coincides with a corresponding area of the partial image A2. Similarly, the overlapping area V12 is an area of the partial image A2 and coincides with a corresponding area of the partial image A1. For example, the overlapping region V12 can be obtained as a region where the evaluation function that compares the feature amounts of two partial images shows the maximum degree of coincidence. As the feature amount of the partial image, for example, an edge in the image can be used.

  Similarly, the partial image A2 and the partial image A4 are arranged so that the overlapping region V24 matches. Here, a vector having the image gravity center G2 of the partial image A2 as the start point and the image gravity center G4 of the partial image A4 as the end point is referred to as a position vector P24.

  Similarly, the partial image A1 and the partial image A3 are arranged so that the overlapping area V13 matches. Here, a vector having the image gravity center G1 of the partial image A1 as the start point and the image gravity center G3 of the partial image A3 as the end point is referred to as a position vector P13.

  Further, the partial image A3 and the partial image A4 are arranged so that the overlapping region V34 coincides. Here, a vector having the image gravity center G3 of the partial image A3 as the start point and the image gravity center G4 of the partial image A4 as the end point is referred to as a position vector P34.

International Publication No. 2005/024723 Pamphlet

  The position P4 of the image center of gravity G4 of the partial image A4 that is the end point of the combined vector obtained by combining the position vector P12 and the position vector P24 described above is the partial image that is the end point of the combined vector obtained by combining the position vector P13 and the position vector P34. It may be different from the position Q4 of the image centroid G4 of A4.

  This is because an error occurs when the overlapping area is obtained using the evaluation function. For example, the area of the partial image A1 that matches the corresponding area of the partial image A2 and the area of the partial image A2 that matches the corresponding area of the partial image A1 are the same part of the subject. Although it is a photographed image area, the luminance or chromaticity of the pixels does not always match completely. Therefore, the overlap region V12 may be different from the true overlap region.

  Accordingly, when a composite image is generated by combining a plurality of partial images, there may be a problem that a deviation occurs.

  Therefore, an object of the present specification is to provide a method for synthesizing an image that reduces the deviation of the synthesized image.

  Another object of the present specification is to provide a program for synthesizing an image that reduces the deviation of the synthesized image in order to solve the above-described problem.

  Furthermore, in this specification, in order to solve the above-described problem, an object of the present invention is to provide an image synthesizing apparatus that reduces the deviation of the synthesized image.

  According to the method for synthesizing images disclosed in the present specification, a subject is a partial image that is captured by being divided into a plurality of images, and two partial images that are captured adjacent to each other are overlapped by capturing the same portion of the subject. Using a plurality of partial images having areas, for each pair of two partial images photographed adjacent to each other, the two partial images are arranged so that the overlapping areas coincide with each other. A plurality of partial images so as to correspond to the first step of generating a virtual spring having the distance between the positions of the image centroids as a natural length, and the positional relationship between adjacent partial images. In the second step of generating a spring model in which the image centroids of two adjacent partial images are connected by virtual springs, and the difference between the length of the virtual spring connected to the image centroid and the natural length. The third step of calculating the motion of the center of gravity of the image where the restoring force of the virtual spring worked and determining the equilibrium position, which is the position of the center of gravity of the image when the restoring force of all the virtual springs is balanced, and each of the partial images Is arranged such that the image center of gravity is located at the equilibrium position, and a synthesized image is generated.

  In addition, according to the method for synthesizing images disclosed in the present specification, a subject is a partial image that is captured by being divided into a plurality of images, and two adjacent partial images are captured at the same portion of the subject. Using two or more partial images having overlapping areas, two partial images are arranged so that the overlapping areas coincide with each other for each pair of two partial images photographed adjacent to each other. The first step of generating a virtual spring having the distance between the positions of the image centroids of each image as a natural length, and a plurality of parts so as to correspond to the positional relation when the adjacent partial images are photographed In a state where each image is arranged, a second step of generating a spring model in which the image centroids of two adjacent partial images are connected by a virtual spring, and the length and natural length of the virtual spring connected to the image centroid A third step of calculating the motion of the center of gravity of the image where the restoring force of the virtual spring based on is calculated, and obtaining an equilibrium position that is the position of the center of gravity of the image in a state where the restoring forces of all the virtual springs are balanced, and a plurality of partial images A fourth step of generating a composite image by arranging each so that the image center of gravity is located at the equilibrium position is executed by the computer.

  In addition, according to the program for synthesizing images disclosed in the present specification, a subject is a partial image that is captured by being divided into a plurality of images, and two adjacent images that are captured adjacently capture the same portion of the subject. Using two or more partial images having overlapping areas, two partial images are arranged so that the overlapping areas coincide with each other for each pair of two partial images photographed adjacent to each other. The first step of generating a virtual spring having the distance between the positions of the image centroids of each image as a natural length, and a plurality of parts so as to correspond to the positional relation when the adjacent partial images are photographed A second step of generating a spring model in which the image centroids of two adjacent partial images are connected by virtual springs in a state where each image is arranged, and the length and naturalness of the virtual spring connected to the image centroid A third step of calculating the motion of the center of gravity of the image where the restoring force of the virtual spring based on the difference between and the equilibrium position, which is the position of the center of gravity of the image in a state where the restoring forces of all the virtual springs are balanced, and a plurality of steps Each of the partial images is arranged so that the center of gravity of the image is positioned at the equilibrium position, and the computer is caused to execute the fourth step of generating a composite image.

  Further, according to the image composition device disclosed in the present specification, the subject is a partial image obtained by dividing the subject into a plurality of images, and two partial images taken adjacent to each other overlap each other by photographing the same portion of the subject. A storage unit that stores a plurality of partial images having an area, and a plurality of partial images are read from the storage unit, and an overlapping area matches each pair of two partial images photographed adjacent to each other. In this way, the two partial images are arranged in such a manner that a virtual spring having a natural length as the distance between the positions of the center of gravity of each of the two partial images is generated, and the positional relationship between the adjacent partial images is photographed. Correspondingly, in a state where each of the plurality of partial images is arranged, a spring model in which the image centroids of two adjacent partial images are connected by a virtual spring is generated, and the length of the virtual spring connected to the image centroid Calculate the motion of the center of gravity of the image where the restoring force of the virtual spring based on the difference from the length was calculated, and find the equilibrium position that is the position of the center of gravity of the image when the restoring force of all the virtual springs is balanced. Each of the images includes a calculation unit that arranges the images so that the center of gravity of the image is located at the equilibrium position and generates a combined image, and an output unit that outputs the combined image.

  According to the method for synthesizing the image disclosed in the present specification described above, the shift of the synthesized image can be reduced.

  Further, according to the above-described program for synthesizing images disclosed in the present specification, it is possible to reduce the deviation of the synthesized image.

  Furthermore, according to the image composition device disclosed in the present specification described above, the shift of the composite image can be reduced.

It is a figure explaining combining a partial image by the conventional example. It is a block diagram showing an image composition device indicated in this specification. It is a figure which shows the some partial image synthesize | combined by the image synthesizer disclosed in this specification. It is a flowchart (the 1) explaining operation | movement of an image synthesizing | combining apparatus. It is a figure explaining step S10. It is a figure explaining step S12. It is a figure which shows a spring model. It is a flowchart (the 2) explaining operation | movement of an image synthesizing | combining apparatus. It is a figure explaining step S22 and S24. It is a figure explaining arrange | positioning so that an image gravity center may be located in an equilibrium position, and producing a synthesized image for each of two partial images. It is a flowchart (the 3) explaining operation | movement of an image synthesizing | combining apparatus. It is FIG. (1) explaining correction | amendment of a partial image. It is FIG. (2) explaining correction | amendment of a partial image. FIG. 6 is a third diagram illustrating correction of a partial image. FIG. 6 is a diagram (part 4) for explaining correction of a partial image. FIG. 10 is a diagram (part 5) illustrating correction of a partial image. It is FIG. (6) explaining correction | amendment of a partial image.

  Hereinafter, a preferred embodiment of an image composition device disclosed in this specification will be described with reference to the drawings. However, the technical scope of the present invention is not limited to these embodiments, but extends to the invention described in the claims and equivalents thereof.

  FIG. 2 is a block diagram showing an image composition apparatus disclosed in this specification.

  An image composition device 10 (hereinafter also simply referred to as device 10) of the present embodiment executes a storage unit 12 that stores a plurality of partial images and a predetermined program, and a predetermined program stored in the storage unit 12, And a calculation unit 11 that generates a composite image based on a plurality of partial images stored in the unit 12.

  The apparatus 10 also includes a display unit 13 that displays the generated composite image and an output unit 15 such as a printer that outputs the generated composite image. The output unit 15 may have the function of the display unit 13.

  In addition, the apparatus 10 includes an input unit 14 such as a keyboard or a mouse, and a communication unit 16 for communicating with the outside such as a network.

  The apparatus 10 can be configured using, for example, a computer such as a server or a personal computer, or a state machine.

  FIG. 3 is a diagram illustrating a plurality of partial images combined by the image combining apparatus disclosed in this specification.

  The plurality of partial images A1 to An are obtained by shooting the subject B divided into a plurality of parts. The plurality of partial images A1 to An can be taken using, for example, an array camera. Each of the two partial images photographed adjacent to each other has an overlapping region V where the same portion of the subject B is photographed. In the present embodiment, there are four or more partial images.

  For example, the partial image Ai has an overlapping region V with the partial images adjacent to the left, top, right, and bottom.

  A preferred embodiment of the operation of the device 10 will now be described below with reference to the drawings.

  FIG. 4 is a flowchart (part 1) for explaining the operation of the image composition apparatus.

  In the apparatus 10, the calculation unit 11 performs a process of each step by executing a predetermined program stored in the storage unit 12.

  First, in step S <b> 10, a plurality of partial images A <b> 1 to An stored in the storage unit 12 are read out so that the overlapping region V matches each pair of two partial images photographed adjacently. Two partial images can be arranged.

  FIG. 5 shows a state in which two partial images Ai and Aj photographed adjacent to each other are arranged so that their overlapping regions V coincide with each other.

  For example, the overlapping region V is a region of the partial image Ai and is a region that matches the corresponding region of the partial image Aj. Similarly, the overlapping area V is an area of the partial image Aj and coincides with a corresponding area of the partial image Ai. The overlapping region V can be obtained, for example, as a region where the evaluation function that compares the feature amounts of two partial images shows the maximum degree of coincidence. As the feature amount of the partial image, for example, an edge in the image can be used.

  Here, a vector having the image gravity center Gi of the partial image Ai as the start point and the image gravity center Gj of the partial image Aj as the end point is referred to as an optimum position vector Pij.

  Then, a virtual spring S having a natural length as a distance L between the positions of the image centroids of the two partial images Ai and Aj is generated. Let the spring constant of the virtual spring S be k. The distance L corresponds to the size of the optimum position vector Pij.

  The virtual spring S is generated for each pair of two partial images photographed adjacent to each other.

  Next, in step S12, each of the plurality of partial images is arranged in a state of being arranged so that the positional relationship between the adjacent partial images corresponds to the positional relationship at the time of photographing.

  FIG. 6 shows an example of a state in which each of a plurality of partial images is arranged so that the positional relationship between adjacent partial images corresponds to the positional relationship at the time of photographing.

  The plurality of partial images A1 to An are arranged on the reference coordinates S1. The reference coordinate S1 has an origin O1, an x1 axis, and a y1 axis.

  First, among the plurality of partial images A1 to An, the partial images constituting the top row of the composite image are arranged so that the overlapping areas of the two adjacent partial images coincide with each other. In the example illustrated in FIG. 6, the partial image A1 and the partial image A2 are arranged so that the overlapping region V12 matches. Further, the partial image A2 and the partial image A3 are arranged so that the overlapping region V23 matches. Thereafter, the partial images constituting the top row of the composite image are arranged in the same manner.

  Next, each of the partial images constituting the uppermost row of the composite image and the adjacent partial images below are arranged so that the overlapping areas of the two partial images coincide. In the example shown in FIG. 6, the partial image A1 and the partial image Ak are arranged so that the overlapping area V1k matches. Further, the partial image A2 and the partial image Ak + 1 are arranged so that the overlapping region V2k + 1 matches.

  In the same manner, the partial image constituting the uppermost row of the composite image to the partial image constituting the lowermost row of the composite image are arranged on the reference coordinate S1.

  On the reference coordinates S1, the position of the partial image is expressed using the reference coordinates. For example, the image center of gravity Gi of the partial image Ai is represented by a position Ci on the reference coordinate S1. The position Ci is defined by coordinates x1 and y1.

  Each partial image has a partial coordinate S2 indicating a position in the partial image. The partial coordinate S2 has an origin O2, an x2 axis, and a y2 axis. As the origin O2, for example, if the partial image is a rectangle, it can be the top left vertex of the partial image. The position in the partial image can be expressed using this partial coordinate S2.

  The arrangement of the partial images shown in FIG. 6 is an example. The plurality of partial images may be arranged so as to be different from those in FIG. 6 as long as the positional relationship between the adjacent partial images corresponds to the positional relationship when the partial images are captured. For example, one partial image and another partial image adjacent to the one partial image may be arranged apart from each other. However, it is preferable to arrange adjacent images so as to overlap each other in order to shorten the calculation time described later.

  And the spring model which connected between the image gravity centers of two adjacent partial images with the virtual spring S is produced | generated.

  FIG. 7 is a diagram showing a spring model.

  Image centroids of adjacent partial images are combined by a virtual spring S to generate a spring model. In this spring model, the image center of gravity of the partial image is regarded as a mass point having a mass m. The image centroids of the adjacent partial images are combined by a virtual spring having a natural length generated in step S10.

  At this time, the position of the center of gravity of each image is determined by the position at which each partial image is arranged on the reference coordinate S1, as shown in FIG.

  For example, the position on the reference coordinate S1 of the image gravity center G1 of the partial image A1 is C1. The position on the reference coordinate S1 of the image gravity center G2 of the partial image A2 is C2. Similarly, the position on the reference coordinate S1 of the image gravity center Gi of the partial image Ai is Ci. Further, the position on the reference coordinate S1 of the image gravity center Gn of the partial image An is Cn.

  Here, the distance between the position Ci of the image gravity center Gi and the position Cj of the adjacent image gravity center Gj is not necessarily the natural length of the virtual spring S. This is because the partial image Ai and the partial image Aj are not arranged so that their overlapping areas coincide with each other.

  In the spring model of this embodiment, the spring constant k of all virtual models S is the same. Further, in the spring model of the present embodiment, all the image centroids have the same mass.

  Note that the partial image may be weighted by changing the value of the spring constant or mass. By adding weight to the partial image, for example, the calculation time described later can be shortened.

  Next, in step S14, the motion of the image center of gravity where the restoring force of the virtual spring based on the difference between the length of the virtual spring connected to the image center of gravity and the natural length is calculated, and the restoring forces of all the virtual springs are balanced. An equilibrium position, which is the position of the center of gravity of the image in the touched state, is obtained.

  FIG. 8 is a flowchart (part 2) for explaining the operation of the image composition apparatus. FIG. 8 shows the detailed processing of step S14.

  As described above, the distance between the two image centroids coupled by the virtual spring S is not necessarily the natural length of the virtual spring S. When the distance between the two image centroids is different from the natural length, a restoring force by the virtual spring S acts between the two image centroids. The center of gravity of the image where the restoring force is applied moves and changes its position. Therefore, between step S20 and step S34, the motion of the image centroid is calculated for all the partial images A1 to An until the restoring forces of all the virtual springs are balanced.

  First, in step S22, the optimal position vector Pij (see FIG. 5) between the image centroid Gi of the partial image Ai and the image centroid Gj of the adjacent partial image Aj, and the current image centroid Gi and the image centroid Gj. Based on the position vector Cij, a force vector fij acting on the image gravity center Gi is obtained.

  As shown in FIG. 9, the position vector Cij is a position vector having a position Ci of the image center of gravity Gi as a start point and a position Cj of the image center of gravity Gj as an end point.

  In the example shown in FIG. 9, the image centroid Gi is coupled to the image centroids Gj1, Gj2, Gj3, and Gj4 adjacent to the left, top, right, and bottom of the image centroid Gi by virtual springs S. Note that the natural lengths of the four virtual springs S coupled to the image gravity center Gi are not necessarily the same.

Thus, the following restoring force vector fijm acts between the image gravity center Gi and each image gravity center Gjm (m is an integer of 1 to 4).
fijm = k (Pijm-Cijm)

  Accordingly, the resultant force vector Fi of the four restoring force vectors acts on the image gravity center Gi.

  Next, in step S24, the magnitude of the resultant force vector Fi generated at the image gravity center Gi is obtained.

  Next, in step S26, it is determined whether or not the magnitude of the resultant vector Fi is equal to or less than a threshold value. If the magnitude of the resultant vector Fi is equal to or smaller than the threshold value, the restoring force with respect to the image centroid Gi is in a balanced state, so the position of the image centroid Gi is determined to be at the equilibrium position Oi, and the process proceeds after step S32.

  On the other hand, if the magnitude of the resultant vector Fi is larger than the threshold value, the process proceeds to step S28.

  Next, in step S28, an acceleration vector αi of the image center of gravity Gi is obtained from the resultant force vector Fi.

Next, in step S30, a velocity vector vi of the image center of gravity Gi is obtained from the acceleration vector. The velocity vector vi is obtained by the following equation using the velocity vector vit of the current image gravity center Gi and the acceleration vector αi.
vit + 1 = vit + Δt · αi
Here, the velocity vector vit + 1 is a velocity vector after time Δt with respect to the velocity vector vit of the current image gravity center Gi.

Next, in step S32, a position vector ci of the image gravity center Gi is obtained from the velocity vector. The position vector ci is obtained by the following expression using the position vector cit of the current image center of gravity Gi and the velocity vector vi.
cit + 1 = cit + Δt · vi
Here, the position vector cit + 1 is a position vector after a time Δt with respect to the current position vector cit of the image gravity center Gi.

  The processes in steps S22 to S32 are repeated for all partial images until the determination in step S26 becomes Yes. For example, if the determination in step S26 is No in one partial image, the processes in steps S22 to S32 are performed again for all partial images.

  In this way, the equilibrium positions O1 to On, which are the positions of the image centroids G1 to Gn in a state where the restoring forces of all the virtual springs are balanced, are obtained by the processing of steps S20 to S30. Here, the equilibrium positions O1 to On are the positions of the end points of the position vectors c1 to cn on the reference coordinate S1. Then, the process proceeds to step S16.

  Next, in step S16, the plurality of partial images A1 to An are arranged on the reference coordinates S1 so that the image centroids G1 to Gn are positioned at the equilibrium positions O1 to On, and a composite image is generated.

  As described above, the apparatus 10 moves the positions of the image centroids G1 to Gn to the equilibrium positions O1 to On so that the restoring forces of all the virtual springs are balanced. Dispersing the synthesized image as a whole avoids the local concentration of misalignment.

  FIG. 10 is a diagram for explaining that a composite image is created by arranging two partial images so that the center of gravity of the image is located at an equilibrium position.

  The partial image Ajo is obtained by arranging the partial image Aj on the reference coordinates S1 so that the image gravity center Gj is located at the equilibrium position Oj with respect to the adjacent partial image Ai. Here, the partial image Ai is arranged on the reference coordinate S1 so that the image gravity center Gi is located at the equilibrium position Oi. A position vector starting from the equilibrium position Oi (Gi) and ending at the equilibrium position Oj is hereinafter also referred to as an equilibrium position vector Oij. In this way, the partial images A1 to An are arranged on the reference coordinates S1 so that the image centroids G1 to Gn are positioned at the equilibrium positions O1 to On, and a composite image is generated.

  In FIG. 10, the partial image Ajp is arranged on the reference coordinate S <b> 1 so that the overlapping area Vp matches the partial image Aj with respect to the adjacent partial image Ai. The optimum position vector Pij is a position vector having the equilibrium position Oi (Gi) as the start point and the image gravity center Gj as the end point.

  As shown in FIG. 10, when the image center of gravity Gi and the equilibrium position Oi are matched, the equilibrium position vector Oij and the optimal position vector Pij may not match.

  In the overlapping area Vo between the partial image Ai and the partial image Ajo, there is a possibility that the image shift is larger than in the overlapping area Vp between the partial image Ai and the partial image Ajp.

  Thus, by dispersing the local shift that occurs in the composite image over the entire composite image, on the other hand, there may be a region in the composite image where the image shift is large, such as the overlapping region Vo. is there.

  Therefore, the device 10 corrects the partial image to reduce the deviation that may occur in the overlapping region Vo. Then, the apparatus 10 generates a composite image using the corrected partial image.

  Next, processing for correcting the partial image by the apparatus 10 will be described below with reference to the drawings.

  FIG. 11 is a flowchart (part 3) for explaining the operation of the image composition apparatus.

  First, in step S40, as shown in FIG. 12, two adjacent images Ai and Aj are placed on the reference coordinate S1 so that the image gravity centers Gi and Gj are located at the equilibrium positions Oi and Oj. Line up. Then, an equilibrium position vector Oij having the image centroid Gi of the partial image Ai as the start point and the image centroid Gj of the partial image Aj adjacent to the partial image as the end point is obtained. Here, the partial image Ajo is obtained by arranging the partial image Aj on the reference coordinate S1 so that the image gravity center Gj is located at the equilibrium position Oj with respect to the adjacent partial image Ai.

  Further, two partial images Ai and Aj photographed adjacent to each other are arranged on the reference coordinate S1 so that the overlapping region Vp matches. Then, an optimum position vector Pij having the position of the image center of gravity Gi of the partial image Ai as the start point and the position of the image center of gravity Gj of the partial image Aj adjacent to the partial image Ai as the end point is obtained. Here, the partial image Ajp is the partial image Aj arranged on the reference coordinate S1 so that the overlapping region Vp matches the adjacent partial image Ai.

  Then, a difference vector D which is a difference between the equilibrium position vector Oij and the optimum position vector Pij is obtained.

  Next, in step S42, as shown in FIG. 12, the position of the intersection point Xl between the optimum position vector Pij and the end of the partial image Ai is obtained. The intersection point Xl is a point where the side on the left partial image Aj side among the four sides constituting the rectangular partial image Ai intersects with the optimum position vector Pij.

  Next, in step S44, as shown in FIG. 12, the position of the intersection point Xl is changed from the ratio between the distance L1 between the starting point Gi of the optimum position vector Pij and the intersection point X1 and the magnitude L2 of the optimum position vector Pij. The corrected intersection Xlo is obtained by moving in the direction of the difference vector by the product of the magnitude of the vector D (L1 / L2 · | D |).

  A position vector having the intersection point Xl as the start point and the corrected intersection point Xlo as the end point is referred to as a position vector El.

  As described above, the corrected intersection Xlo for correcting the partial image Ai with respect to the partial image Aj adjacent to the left (hereinafter also referred to as the partial image L adjacent to the left) is obtained.

  As shown in FIG. 13, similarly, a corrected intersection point Xto for correcting the partial image Ai with respect to the partial image T adjacent above is obtained. A position vector having the intersection point Xt as the start point and the corrected intersection point Xto as the end point is referred to as a position vector Et.

  Similarly, a corrected intersection Xro for correcting the partial image Ai with respect to the partial image R adjacent to the right is obtained. A position vector starting from the intersection point Xr and ending at the corrected intersection point Xro is referred to as a position vector Er.

  Further, similarly, a corrected intersection point Xbo for correcting the partial image Ai with respect to the lower adjacent partial image B is obtained. A position vector having the intersection point Xb as a start point and the corrected intersection point Xbo as an end point is referred to as a position vector Eb.

  In addition, the end point when the position vector El and the combined vector Etl of the Et are arranged starting from the top left vertex Xtl of the rectangular partial image Ai is referred to as a corrected vertex Xtlo.

  Further, the end point when the position vector Et and the combined vector Etr of the Er are arranged starting from the upper right vertex Xtr of the rectangular partial image Ai is referred to as a corrected vertex Xtro.

  In addition, the end point when the position vector Er and the combined vector Ebr of the Eb are arranged starting from the lower right vertex Xbr of the rectangular partial image Ai is referred to as a corrected vertex Xbro.

  Furthermore, the end point when the position vector Eb and the combined vector Ebl of the El are arranged starting from the lower left vertex Xbl of the rectangular partial image Ai is referred to as a corrected vertex Xblo.

  Then, as shown in FIG. 14, the corrected vertices Xtlo, Xtro, Xbro, Xblo and the corrected intersections Xlo, Xto, Xro, Xbo are connected to deform the partial image Ai and the corrected partial image Hi. Is obtained.

  Next, for the corrected partial image Hi, the image centroid Gi and the corrected vertex Xtlo and the corrected intersection points Xlo and Xto are surrounded by the polygon html, and the image centroid Gi and the corrected vertex Xtro are corrected. Polygon htr surrounded by intersections Xto and Xro, image center of gravity Gi and corrected vertex Xbro and polygon hbr surrounded by corrected intersections Xro and Xbo, image center of gravity Gi and corrected vertex Xblo and correction A polygon hbl surrounded by the intersected points Xbo and Xlo is generated.

  At this stage, the corrected partial image Hi is generated as a polygon composed of the image center of gravity Gi, the corrected vertices Xtro, Xtro, Xbro, Xblo and the corrected intersections Xlo, Xto, Xro, Xbo. However, pixels are not yet embedded in each polygon.

  The partial image correction described above is performed on all partial images.

  Here, correction of the partial image constituting the top row of the composite image among the plurality of partial images A1 to An will be described below with reference to FIG.

  In FIG. 15, the partial image H1 corrected by deforming the partial image A1 is an image located in the upper left corner of the plurality of partial images. The corrected partial image H1 includes a polygon h1tl located at the upper left, a polygon h1tr located at the upper right, a polygon h1br located at the lower right, and a polygon h1bl located at the lower left.

  The polygon h1tl positioned at the upper left has no partial image adjacent to the left and above the partial image A1 before correction, and thus has not been corrected, and has the shape of the corresponding region of the partial image A1. .

  The polygon h1tr located in the upper right is related to the partial image R adjacent to the right with respect to the partial image A1 before correction, and using the equilibrium position vector and the optimal position vector, as described in FIG. The positions of the intersections and vertices are corrected and generated.

  The polygon h1br located at the lower right is the equilibrium position in the same manner as described with reference to FIG. 12 in relation to the partial image R adjacent to the right and the partial image B adjacent below the partial image A1 before correction. Using the vector and the optimum position vector, the positions of the intersection and the vertex are corrected and generated.

  The polygon h1bl located in the lower left is related to the partial image B adjacent to the lower side of the partial image A1 before correction, and using the equilibrium position vector and the optimum position vector, as described in FIG. The positions of the intersections and vertices are corrected and generated.

  Next, a partial image Hk which is a partial image constituting the top row of the composite image and is corrected by deforming the partial image Ak having the partial images adjacent to the left and right will be described below.

  The corrected partial image Hk includes a polygon hktl located at the upper left, a polygon hktr located at the upper right, a polygon hkbr located at the lower right, and a polygon hkbl located at the lower left.

  The polygon hktl located in the upper left is related to the partial image L adjacent to the left with respect to the partial image Ak before correction using the equilibrium position vector and the optimum position vector, as described in FIG. The positions of the intersections and vertices are corrected and generated.

  The polygon hktr located on the upper right side is related to the partial image R adjacent to the right side with respect to the partial image Ak before correction using the equilibrium position vector and the optimum position vector, as described in FIG. The positions of the intersections and vertices are corrected and generated.

  The polygon hkbr located at the lower right is a balanced position similar to that described with reference to FIG. 12 in relation to the partial image R adjacent to the right and the partial image B adjacent below the partial image Ak before correction. Using the vector and the optimum position vector, the positions of the intersection and the vertex are corrected and generated.

  The polygon hkbl located at the lower left is a balanced position vector similar to that described with reference to FIG. 12 in relation to the partial image L adjacent to the left and the partial image B adjacent below the partial image Ak before correction. And the position of the intersection and the vertex are corrected and generated using the optimum position vector.

  As described above, the adjacent images constituting the bottom image of the composite image, the partial image constituting the leftmost column, and the partial image constituting the rightmost column are also adjacent to each other. Therefore, using the equilibrium position vector and the optimum position vector, the positions of the intersection and the vertex are corrected, and a corrected partial image is generated.

  Next, a process of embedding pixels in each polygon for the corrected partial image Hi will be described below with reference to FIGS.

  First, as shown in FIG. 16, each polygon is divided into a predetermined number of meshes for the corrected partial image Hi.

  In the example shown in FIG. 16, the corrected partial image Hi has a polygon htl located at the upper left, a polygon htr located at the upper right, a polygon hbr located at the lower right, and a polygon hbl located at the lower left.

  The polygon htl located in the upper left is divided into 4 × 4 meshes with each side being divided into four equal intervals. Although not shown, the other three polygons are similarly divided into 4 × 4 meshes. The polygon may be divided into a number other than 4 × 4.

  Next, as shown in FIG. 17, with respect to the partial image Ai before correction of the partial image Hi, the polygon atl surrounded by the image center of gravity Gi, the vertex Xtl, and the intersection points Xl and Xt, the image gravity center Gi, and the vertex Xtr And a polygon atr surrounded by the intersections Xt and Xr, a polygon abr surrounded by the image centroid Gi, the vertex Xbr and the intersections Xr and Xb, and an image centroid Gi, the vertex Xbl and the intersections Xb and Xl. A polygon abl is generated.

  And with respect to the partial image Ai, each polygon is divided | segmented into a predetermined number of meshes similarly to the corrected partial image Hi.

  In the example shown in FIG. 17, the polygon atl located at the upper left is divided into 4 × 4 meshes, with each side being divided into four equal intervals, like the polygon htl of the corrected partial image Hi. . Although not shown, the other three polygons of the corrected partial image Hi are similarly divided into 4 × 4 meshes.

  Then, the pixels of each mesh of the polygon atl are pasted while interpolating with respect to the corresponding mesh of the polygon html using a known technique.

  In the example illustrated in FIG. 17, pixels of the mesh az of the polygon atl are pasted while being interpolated with respect to the mesh hz corresponding to the polygon htl. Similarly, the pixels of each mesh of the polygon atl are pasted to the corresponding mesh of the polygon html while interpolating.

  Similarly, the pixels of the corresponding polygon mesh of the partial image Ai are pasted to each mesh of the other three polygons of the corrected partial image Hi while interpolating.

  In this way, pixels are embedded in the corrected partial image Hi.

  The same processing as described above is performed on all corrected partial images.

  Then, the corrected partial images H1 to Hn are arranged on the reference coordinates S1 so that the image gravity centers G1 to Gn are located at the equilibrium positions O1 to On, and a composite image is generated.

  According to the image composition apparatus of the present embodiment described above, a composite image with reduced image deviation can be obtained.

  In the present invention, the method for synthesizing the image, the program for synthesizing the image, and the image synthesizing apparatus of the above-described embodiment can be appropriately changed without departing from the gist of the present invention.

  For example, in the embodiment described above, a rectangular partial image is used, but the partial image may not be rectangular. For example, when the partial image is obtained by photographing a curved surface, it is preferable to correct the distortion of the partial image so as to be developed on a plane and generate a composite image using the corrected partial image. In this case, the partial image corrected so as to develop the distortion of the partial image on a plane is not rectangular. A composite image may be generated using the partial image corrected in this way.

DESCRIPTION OF SYMBOLS 10 Image composition apparatus 11 Computation part 12 Storage part 13 Display part 14 Input part 15 Output part 16 Communication part A1-An Partial image H1-Hn Corrected partial image B Subject V Overlapping area Gi Image gravity center of partial image i m Image gravity center Mass S Virtual spring k Spring constant Pij Optimal position vector Cij Position vector Oij Equilibrium position vector Oi Equilibrium position of image centroid of partial image i D Difference vector

Claims (8)

Adjacent using a plurality of partial images obtained by dividing a subject into a plurality of images and having two overlapping images obtained by photographing two images of the same subject. A virtual spring having two partial images arranged in such a manner that the overlapping regions coincide with each other, and the distance between the positions of the image centroids of the two partial images as a natural length. A first step of generating
A virtual spring connects the image centroids of two adjacent partial images with each of the partial images arranged so that the positional relationship between adjacent partial images corresponds to the positional relationship at the time of shooting. A second step of generating a spring model
The position of the image center of gravity when the restoring force of all virtual springs is balanced by calculating the motion of the image center of gravity where the restoring force of the virtual spring is based on the difference between the natural length and the length of the virtual spring connected to the image center of gravity. A third step for determining an equilibrium position of
Two partial images photographed adjacent to each other are arranged so that the center of gravity of each image is located at the equilibrium position, and the center of gravity of one partial image is used as the starting point, and the center of gravity of another partial image adjacent to the one partial image The first position vector with ending point and two partial images taken adjacent to each other are arranged so that the overlapping areas coincide with each other, and the position of the center of gravity of one partial image is set as the starting point and is adjacent to the one partial image. A fourth step of obtaining a difference vector that is a difference between the second position vector having the end point at the position of the image centroid of another partial image and deforming one partial image based on the difference vector;
A fifth step of arranging each of the plurality of partial images so that the center of gravity of the image is located at an equilibrium position, and generating a composite image;
A method comprising: Find the position of the intersection of the second position vector and the edge of one partial image,
The position of the intersection point is moved in the direction of the difference vector by the product of the ratio between the distance between the starting point of the second vector and the intersection point and the magnitude of the second position vector and the magnitude of the difference vector. The method according to claim 1 , wherein the partial image is deformed. Spring model A method according to claim 1 or 2 is the spring constant of all the virtual models are the same. Spring model, the method according to any one of claim 1 to 3 mass of all the image center of gravity is the same. The number of partial images is four or more, The method as described in any one of Claims 1-4 . Adjacent using a plurality of partial images obtained by dividing a subject into a plurality of images and having two overlapping images obtained by photographing two images of the same subject. A virtual spring having two partial images arranged in such a manner that the overlapping regions coincide with each other, and the distance between the positions of the image centroids of the two partial images as a natural length. A first step of generating
A virtual spring connects the image centroids of two adjacent partial images with each of the partial images arranged so that the positional relationship between adjacent partial images corresponds to the positional relationship at the time of shooting. A second step of generating a spring model
The position of the image center of gravity when the restoring force of all virtual springs is balanced by calculating the motion of the image center of gravity where the restoring force of the virtual spring is based on the difference between the natural length and the length of the virtual spring connected to the image center of gravity. A third step for determining an equilibrium position of
Two partial images photographed adjacent to each other are arranged so that the center of gravity of each image is located at the equilibrium position, and the center of gravity of one partial image is used as the starting point, and the center of gravity of another partial image adjacent to the one partial image The first position vector with ending point and two partial images taken adjacent to each other are arranged so that the overlapping areas coincide with each other, and the position of the center of gravity of one partial image is set as the starting point and is adjacent to the one partial image. A fourth step of obtaining a difference vector that is a difference between the second position vector having the end point at the position of the image centroid of another partial image and deforming one partial image based on the difference vector;
A fifth step of arranging each of the plurality of partial images so that the center of gravity of the image is located at an equilibrium position, and generating a composite image;
A method that is executed by a computer. Adjacent using a plurality of partial images obtained by dividing a subject into a plurality of images and having two overlapping images obtained by photographing two images of the same subject. A virtual spring having two partial images arranged in such a manner that the overlapping regions coincide with each other, and the distance between the positions of the image centroids of the two partial images as a natural length. A first step of generating
A virtual spring connects the image centroids of two adjacent partial images with each of the partial images arranged so that the positional relationship between adjacent partial images corresponds to the positional relationship at the time of shooting. A second step of generating a spring model
The position of the image center of gravity when the restoring force of all virtual springs is balanced by calculating the motion of the image center of gravity where the restoring force of the virtual spring is based on the difference between the natural length and the length of the virtual spring connected to the image center of gravity. A third step for determining an equilibrium position of
Two partial images photographed adjacent to each other are arranged so that the center of gravity of each image is located at the equilibrium position, and the center of gravity of one partial image is used as the starting point, and the center of gravity of another partial image adjacent to the one partial image The first position vector with ending point and two partial images taken adjacent to each other are arranged so that the overlapping areas coincide with each other, and the position of the center of gravity of one partial image is set as the starting point and is adjacent to the one partial image. A fourth step of obtaining a difference vector that is a difference between the second position vector having the end point at the position of the image centroid of another partial image and deforming one partial image based on the difference vector;
A fifth step of arranging each of the plurality of partial images so that the center of gravity of the image is located at an equilibrium position, and generating a composite image;
A program that causes a computer to execute. A storage unit for storing a plurality of partial images in which a subject is captured by dividing the subject into a plurality of images, and two adjacent partial images captured by capturing the same portion of the subject When,
Read multiple partial images from the storage unit,
For each pair of two partial images photographed adjacent to each other, two partial images are arranged so that the overlapping regions coincide with each other, and a virtual length having a distance between the positions of the image centroids of the two partial images as a natural length Generate springs,
A virtual spring connects the image centroids of two adjacent partial images with each of the partial images arranged so that the positional relationship between adjacent partial images corresponds to the positional relationship at the time of shooting. Generated spring model,
The position of the image center of gravity when the restoring force of all virtual springs is balanced by calculating the motion of the image center of gravity where the restoring force of the virtual spring is based on the difference between the natural length and the length of the virtual spring connected to the image center of gravity. Find the equilibrium position
Two partial images photographed adjacent to each other are arranged so that the center of gravity of each image is located at the equilibrium position, and the center of gravity of one partial image is used as the starting point, and the center of gravity of another partial image adjacent to the one partial image The first position vector with ending point and two partial images taken adjacent to each other are arranged so that the overlapping areas coincide with each other, and the position of the center of gravity of one partial image is set as the starting point and is adjacent to the one partial image. Obtaining a difference vector that is a difference between the position of the center of gravity of the other partial image and the second position vector having the end point, and deforming one partial image based on the difference vector;
Arranging each of the plurality of partial images so that the image center of gravity is located at the equilibrium position, and generating a composite image;
An output unit for outputting a composite image;
An image synthesizing apparatus.

Images