JP3947289B2 - Image composition processing apparatus and recording medium recording image composition processing program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to an image composition processing apparatus and a recording medium on which an image composition processing program is recorded, and more specifically, to overlap each other to form one subject image.MinutesThe present invention relates to an image composition processing apparatus for joining together a plurality of pieces of image information taken and a recording medium on which an image composition processing program is recorded.
[0002]
[Prior art]
With a silver salt camera, the same subject is continuously photographed while shifting the direction of the camera at the time of photographing, and when creating a print from a plurality of photographed images exposed on these films, the overlapping position of each image is well known. A technique for obtaining a single panorama print while combining them is known.
[0003]
An application of such a technique to an electronic imaging apparatus using an imaging element such as a CCD has been developed. For example, the one proposed by the present applicant in Japanese Patent Application No. 9-157083 can be mentioned.
[0004]
Japanese Patent Application No. 9-157083 discloses an electronic imaging apparatus capable of recording image information corresponding to a set of images photographed by panoramic photographing on a recording medium, and the electronic device in the panoramic photographing described above. Rotation direction setting means for setting information corresponding to the rotation direction of the general imaging device, rotation direction recording means for recording information corresponding to the rotation direction set by the rotation direction setting means on the recording medium, and the recording Control means for controlling the arrangement of the set of panoramic images based on information corresponding to the rotation direction recorded on the medium, and for performing predetermined image editing on the panoramic images arranged by the control means An electronic imaging apparatus comprising display means for displaying editing information and control means for performing a predetermined editing operation based on the editing information displayed on the display means.
[0005]
A plurality of pieces of image information photographed so as to have overlapping portions to form one subject image by the electronic imaging device as described above are stitched together by the electronic imaging device itself. Although they may be combined, at present, it is practical to use a computer such as a personal computer because of the processing speed and the size of the processing program.
[0006]
Therefore, it is desired to realize a processing apparatus and a processing program that can easily perform such image composition without requiring a user to perform complicated operations.
[0007]
[Problems to be solved by the invention]
However, there are various electronic imaging devices such as a horizontal position where the right hand holding the electronic imaging device is right, a vertical position where the right hand is up, or a vertical position where the right hand is down. Shooting is performed in a posture, and there are various directions such as upward, downward, rightward, leftward, etc. for shifting each image. The processing program becomes complicated, leading to a reduction in processing speed and a shortage of memory, and an increase in the price of the processing apparatus.
[0008]
The present invention has been made in view of the above circumstances, and provides an image composition processing apparatus and a recording medium on which an image composition processing program is recorded that can perform a process of combining and synthesizing a plurality of pieces of image information efficiently. The purpose is that.
[0009]
[Means for Solving the Problems]
  In order to achieve the above object, an image composition processing device according to a first aspect of the present invention overlaps each other to form one subject image.MinutesMultiple pieces of image information taken and,Of these image informationIndicates one of the two directions related to the long side and the two directions related to the short side of the rectangular shooting screen.Shooting direction andThe multiple image informationAccompanying information including information related to shooting order,Recording medium capable of recording the above and the accompanying informationOf shooting directionOn the basis of the, The multiple image information in ascending or descending order of the shooting orderSorting means to rearrange and by this sorting meansAscending or descending orderSorted image information,oneTo be an imageFrom one directionAnd image synthesizing means for synthesizing them together.
[0011]
  In addition2The image composition processing device according to the invention is the above-mentioned first.1In the image composition processing device according to the invention, the image composition means is the above2 directions related to the long side and 2 related to the short side of the rectangular shooting screenThere are four basic image composition processing cases related to the direction, and the other image composition processing cases are reduced to any of these four basic image composition processing cases.
[0012]
  First3The image composition processing device according to the invention is the above-mentioned first.2In the image composition processing apparatus according to the invention, the image composition processing case possessed by the image composition means has one case where the long side of the photographing screen is in the horizontal position during photographing and the long side of the photographing screen is photographed in the four directions. By having two cases that are sometimes in the vertical position, a total of 12 image composition processing cases are provided, and the other eight image composition processing cases except the four basic image composition processing cases Each of the above four basic image composition processing cases results in the same information regarding the shooting direction in the information.
[0013]
  First4The image composition processing device according to the invention is the above-mentioned first.3In the image composition processing device according to the invention, the four basic image composition processing cases are obtained by sequentially joining together a plurality of pieces of image information photographed in the first direction among the four directions by a predetermined algorithm and a composite image. A first basic image composition processing case, and a plurality of pieces of image information photographed in a second direction opposite to the first direction are rearranged in descending order of the photographing order, and then sequentially by the predetermined algorithm. The second basic image compositing processing case to be joined to form a composite image and a plurality of pieces of image information photographed in a third direction orthogonal to the first direction are each rotated 90 degrees in one rotation direction. Later, a third basic image composition processing case that is sequentially stitched together by the predetermined algorithm to form a composite image, and a plurality of pieces of image information photographed in a fourth direction opposite to the third direction are displayed. The fourth basic image composition processing case is rotated by 90 degrees in the same rotation direction and then rearranged in the descending order of the photographing order, and then sequentially joined by the predetermined algorithm to form a composite image And a simple image composition processing case.
[0014]
  First5The image composition processing device according to the invention is the above-mentioned first.4In the image composition processing apparatus according to the invention, the other eight image composition processing cases other than the four basic image composition processing cases are the results of the first to fourth basic image composition processing cases. After any of these operations, a manual rotation operation is possible.
  An image composition processing device according to a sixth invention is the image composition processing device according to the fourth invention, wherein the accompanying information further includes information relating to a shooting posture of the camera. Regarding the other eight image composition processing cases other than the basic image composition processing case, the jointly synthesized image is rotated based on the information related to the photographing posture of the camera in the accompanying information.
[0015]
  An image composition processing device according to a seventh invention is the image composition processing device according to the first invention, wherein the rearranging means comprises:Rearranging the plurality of pieces of image information in ascending or descending order of the shooting orderIt has a function of rotating the plurality of pieces of image information.
[0016]
An image composition processing device according to an eighth invention is the image composition processing device according to the first invention, wherein the image composition processing by the image composition means is the same regardless of the photographing direction and photographing order of the plurality of image information. It includes a part that becomes an algorithm.
[0017]
An image composition processing device according to a ninth invention is the image composition processing device according to the eighth invention, wherein the algorithm joins one image information and another image information to form a composite image, and further targets for joining When there is other image information that becomes, the stitching is performed on the composite image.
[0018]
  An image composition processing program recorded on a recording medium on which an image composition processing program according to the tenth invention is recorded is provided on a computer so as to overlap each other to form one subject image.MinutesA plurality of pieces of image information that have been photographed.Indicates one of the two directions related to the long side and the two directions related to the short side of the rectangular shooting screen.Shooting direction andThe multiple image informationAccompanying information including information related to shooting orderOf the above shooting directionOn the basis of the, The multiple image information in ascending or descending order of the shooting orderRearrange,Ascending or descending orderSorted image information,oneTo be an imageFrom one directionThey are joined together and synthesized.
[0019]
  An image composition processing program recorded on a recording medium on which an image composition processing program according to the eleventh invention is recorded is provided on a computer so as to overlap each other to form one subject image.MinutesA plurality of pieces of image information that have been photographed.Indicates one of the two directions related to the long side and the two directions related to the short side of the rectangular shooting screen.Shooting direction andThe multiple image informationAccompanying information including information related to shooting orderOf the above shooting directionOn the basis of the,UpRotate multiple image informationIn addition, the plurality of pieces of image information are arranged in ascending or descending order of the photographing order.Rearrange,Rotated above and ascending or descendingSorted image information,oneTo be an imageFrom one directionThey are joined together and synthesized.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings.
1 to 23 show an embodiment of the present invention, and FIG. 1 is a diagram showing a configuration of a system to which an image composition processing apparatus is applied.
[0021]
A digital camera 1 which is an electronic image pickup device records image data photoelectrically converted by an image pickup device such as a built-in CCD on a detachable recording medium 2 made of, for example, a flash memory.
[0022]
The recording medium 2 is mounted on a reader unit provided in a personal computer 3 as a computer so that recorded image data is read. The personal computer 3 includes a display 4 for displaying images and other processing information, a keyboard 5 for mainly inputting characters, and a mouse for pointing a position on the screen displayed on the display 4. 6 is connected.
[0023]
Further, a digital recorder 7 for recording a large amount of image data, a printer 8 for printing an image, and the like are connected to the personal computer 3 as necessary.
[0024]
Next, the internal configuration of the personal computer 3 will be described with reference to FIG.
[0025]
This personal computer 3 is a rearrangement means for performing image operation and control in accordance with various other programs as well as joining images according to an image composition processing program, and a CPU 11 that also serves as an image composition means. A main memory 12 serving as a recording medium in which the image composition processing program is loaded when the image composition processing program is loaded, and a recording in which the image composition processing program is recorded, such as a hard disk or a floppy disk. An internal recording medium 13 as a medium, an external port 14 for connecting to various external devices such as the digital recorder 7 and the printer 8, an interface (hereinafter abbreviated as IF) 16 for connecting the display 4, and the keyboard IF17 to connect 5 and mouse 6 , The connector 18 and the IF 15 to which the recording medium 2 is electrically connected, and the CPU 11, the main memory 12, the internal recording medium 13, the external port 14, and the IFs 15, 16, and 17, They are connected to each other via a bus.
[0026]
Next, FIG. 3 is a diagram showing the structure of data recorded on the recording medium 2.
[0027]
The image data photographed by the digital camera 1 is recorded on a recording medium 2 in units of files after being subjected to processing such as compression, and the positions of these image files are recorded on the recording medium 2. A file position table is provided.
[0028]
The recording medium 2 may be a general-purpose one or a dedicated one provided for panoramic photography.
[0029]
The image file includes image data (image information) and header information which is accompanying information including various information related to the image data.
[0030]
The header information includes a file name of the image file, a frame number assigned for each shooting, a shooting date and time, and a shooting that indicates in which of the various shooting modes the digital camera 1 has shot. It is configured to include mode information, identification information to be described later, and index image data for use in index display or the like indicating the outline of the image data.
[0031]
The identification information is information used when a panoramic image is formed by joining together image data of a plurality of frames shot with overlapping portions or adjacent portions. For example, the identification information includes 2-byte data. . That is, the first byte of the identification information stores information indicating the shooting direction of the digital camera 1, that is, the direction in which a plurality of frames constituting one panoramic image are shot. It includes information in a total of four directions, ie, two directions on the long side (left-right direction) and two directions on the short side (vertical direction) of the rectangular shooting screen. Specifically, “1” when the long side is in the right direction, “2” when the long side is in the left direction, “3” when the short side is in the upward direction, and “3” when the short side is in the downward direction. 4 "is recorded.
[0032]
Next, the second byte of the identification information is information indicating how many times the image data of the frame is captured in a panoramic image composed of image data of a plurality of frames. They are allocated in order.
[0033]
Whether or not each frame is a part of the panoramic image can be determined by the presence or absence of this identification information.
[0034]
FIG. 4 is a flowchart showing the main routine of the image composition processing program executed by the personal computer 3.
[0035]
Note that the image composition processing program including this main routine and a subroutine described later is recorded on the internal recording medium 13 as described above, and is loaded into the main memory 12 at the time of execution so as to be sequentially processed by the CPU 11. It has become. The image composition processing program is programmed to have a size that can be recorded in the internal recording medium 13 and a size that can be loaded into the main memory 12.
[0036]
This process is started when the user starts the index image display process by operating the mouse 6 or the keyboard 5 (step S1). That is, the image composition processing program recorded on the internal recording medium 13 is loaded into the main memory 12 and is executed sequentially by the CPU 11.
[0037]
First, an image index as will be described later is displayed, and a display is provided so that the target frame to be stitched together can be easily discriminated from other single shooting frames ( Step S2).
[0038]
When the user selects any of the target frames (step S3), all the target frames constituting the panoramic image including the target frame are automatically detected based on the identification information (step S4). In accordance with each of 12 cases corresponding to the shooting state by the digital camera 1, processing for automatically joining image data and synthesizing one panoramic image is performed according to each of 12 cases described later (step S5). An image is displayed (step S6).
[0039]
After that, if the user is not satisfied with the automatically stitched images, it is possible to further select the stitching process manually, and determine whether this manual process is selected (step S7) If not selected, the process is performed as it is. If selected, a manual process described later is performed (step S8), and the process ends.
[0040]
FIG. 5 is a flowchart showing details of the display determination process in step S2 in FIG.
[0041]
When this display discrimination process is started, header information in the data constituting the image file is first read (step S11).
[0042]
Then, by determining whether the first byte of the identification information in the header information is 1 or more and 4 or less, it is determined whether the image data constitutes a panoramic image (step S12).
[0043]
Here, when the first byte of the identification information is 1 or more and 4 or less, the display as illustrated in FIG. 6 or 7 is performed (step S13).
[0044]
FIG. 6 shows an example in which the display color of the background (frame) of the image is changed.
[0045]
A plurality of areas 22 for displaying each frame are arranged in the image index screen 21, and an image 24 of each frame is displayed in the area 22, and the periphery thereof is a frame 23.
[0046]
In the example of FIG. 6, the second frame and the third frame in the upper stage are the first seam image set, and the fourth and fifth frames in the lower stage are the second seam image group. Therefore, by changing the display color of the background (frame), it is displayed that these images constitute a panoramic image.
[0047]
Although the color is changed here, the pattern of the frame may be changed, or both the color and the pattern may be changed to be distinguishable from other images.
[0048]
FIG. 7 shows an example in which a mark is displayed on the frame.
[0049]
In this example, the mark 25 is displayed at the lower right of the target frame, for example, to display that these images constitute a panoramic image.
[0050]
When step S13 ends or when the first byte of the identification information is less than 1 or greater than 4 in step S12, it is determined whether there is a next frame (next header information) (step S14). If there is, the process returns to step S11 and the same process is repeated for the next frame. If there is no next frame in step S14, the process is exited.
[0051]
FIG. 8 is a flowchart showing the target image automatic detection process in step S4 of FIG.
[0052]
This process appropriately discriminates all the frames to be processed regardless of which of a plurality of frames constituting one panoramic image is selected.
[0053]
This process starts when the user points to and selects one frame image from the image index with the keyboard 5 or mouse 6 (step S21).
[0054]
Then, the user activates the splicing process mode using a menu or an icon (step S22). In addition, this step S22 and said step S21 can also be processed by changing order.
[0055]
First, by determining whether or not the first byte of the identification information in the header information is 1 or more and 4 or less, it is confirmed whether the image data constitutes a panorama image (step S23). If it does not constitute an image, this process is exited as it is.
[0056]
On the other hand, when it is confirmed that the image is a panoramic image, the first byte of the identification information is stored in the storage area A, and the second byte of the identification information is stored in the storage area B and the storage area C, respectively. The frame number (image order) is stored in the storage area P (step S24).
[0057]
Then, after recording the current frame as an image to be spliced (step S25), the header information of the next frame is read (step S26), and it is determined whether the reading is successful (step S27). This step S27 determines whether or not there is a next frame. If there is a next frame, it can be read. If the current frame is the last frame and the next frame does not exist anymore. , Reading is unsuccessful.
[0058]
If the reading is successful, the first byte of the identification information is equal to the value stored in the storage area A, and the second byte of the identification information is a value obtained by adding 1 to the value stored in the storage area C; It is determined whether or not they are equal (step S28). As a result, it is determined whether or not it is a frame constituting the same panoramic image and the next frame that is continuous.
[0059]
If this condition is satisfied, the value stored in the storage area C is incremented (step S29), and the process returns to step S25.
[0060]
If the reading in step S27 is unsuccessful or the condition in step S28 is not satisfied, the value stored in the storage area B is copied to the storage area C (step S30). The header information of the frame number that is one smaller than the frame number stored in P, that is, the header information of the previous frame is read (step S31).
[0061]
Then, it is determined whether or not the reading is successful (step S32). If the reading is successful, the first byte of the identification information is equal to the value stored in the storage area A and the second byte of the identification information is stored in the memory. It is determined whether or not it is equal to a value obtained by subtracting 1 from the value stored in the area C (step S33). As a result, it is determined whether or not the frames constituting the same panoramic image are the previous frames.
[0062]
If this condition is satisfied, the value stored in the storage area C is decremented (step S34), recorded as a joining target image (step S35), and the header information of the next frame is read (step S35). S36), the process returns to step S32.
[0063]
If the reading is unsuccessful in step S32 or if the condition in step S33 is not satisfied, it is determined whether the number of images to be stitched is two or more (step S37). If it does not reach 2, the image composition cannot be performed, so a message to that effect is displayed to the user by a dialog box or the like (step S38), and the process is exited.
[0064]
Further, when the number of images to be stitched is two or more, each case for performing automatic stitching as will be described later based on the value stored in the storage area A. Processing branches to processing (step S39).
[0065]
Next, FIG. 9 shows a basic algorithm of the splicing process performed in each case described above, and will be described with reference to FIG.
[0066]
As will be described later for each case, the image information of a plurality of frames constituting the panoramic image is converted based on the identification information by the CPU 11 as the rearranging means before the joining process is performed. As shown in the uppermost part of FIG. 10, the data are rearranged in a predetermined order from left to right.
[0067]
When the splicing process starts, first, the first image from the left (image 1 in FIG. 10) is read (step S41), and the next right image (image 2 in FIG. 10) is read (step S42). ) A seam calculation process is performed on these two images (step S43), and a result image 1 as shown in FIG. 10 is obtained.
[0068]
Next, it is determined whether or not the next right image still exists (step S44). If there is, the processed result image 1 is set as the left image (step S45), and the process returns to step S42 to continue the right image. Perform the alignment process.
[0069]
If it is determined in step S44 that there is no right image, the process is exited.
[0070]
Thus, in this splicing process, two images are sequentially spliced from the left to obtain a final panoramic image.
[0071]
Next, the automatic processing for each case in step S5 of FIG. 4 will be described with reference to FIGS.
[0072]
FIG. 11 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 1, and (B) a flowchart illustrating processing contents.
[0073]
Case 1 is a case where the subject is held in a horizontal position where the right hand holding the digital camera 1 is on the right, and the subject is photographed in a plurality of pieces in order from the left side to the right side. At this time, since shooting is performed in the right direction on the long side of the shooting screen, the first byte of the identification code (same as identification information, the same applies hereinafter) is 1, and the second byte of the identification code is left to right. Ascending order towards.
[0074]
Therefore, in case 1, as shown in FIG. 11B, the splicing process shown in FIG. 9 is performed as it is (step S51), and the process returns.
[0075]
Next, FIG. 12 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 2, and (B) a flowchart illustrating processing contents.
[0076]
Case 2 is a case where the subject is held in a horizontal position where the right hand holding the digital camera 1 is on the right and the subject is photographed in a plurality of pieces in order from the right side to the left side. At this time, since shooting is performed in the left direction on the long side of the shooting screen, the first byte of the identification code is 2, and the second byte of the identification code is in ascending order from right to left.
[0077]
Therefore, in this case 2, as shown in FIG. 12B, first, the image data is rearranged so that the second byte of the identification information is in descending order (step S52), and then the joint shown in FIG. The alignment process is performed (step S53), and the process returns.
[0078]
FIG. 13 is a diagram showing (A) the direction and order of captured images together with an identification code in case 3, and (B) a flowchart showing the processing contents.
[0079]
Case 3 is a case where the subject is held in a vertical position with the right hand holding the digital camera 1 at the bottom, and the subject is photographed in a plurality of pieces in order from the left side to the right side. At this time, since shooting is performed upward on the short side on the shooting screen, the first byte of the identification code is 3, and the second byte of the identification code is in ascending order from left to right.
[0080]
Therefore, in this case 3, as shown in FIG. 13B, all the processing target images are first rotated 90 degrees clockwise (step S54), and then the splicing process shown in FIG. 9 is performed. (Step S55), the process returns.
[0081]
FIG. 14 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 4, and (B) a flowchart illustrating processing contents.
[0082]
Case 4 is a case where the subject is held in a vertical position where the right hand holding the digital camera 1 is at the bottom, and the subject is photographed in a plurality of pieces in order from the right side to the left side. At this time, since shooting is performed in the downward direction on the short side on the shooting screen, the first byte of the identification code is 4, and the second byte of the identification code is in ascending order from right to left.
[0083]
Accordingly, in this case 4, as shown in FIG. 14B, all the processing target images are first rotated 90 degrees clockwise (step S56), and the image data is set so that the second byte of the identification information is in descending order. Are rearranged (step S57), and then the splicing process shown in FIG. 9 is performed (step S58), and the process returns.
[0084]
In the present embodiment, Case 1 to Case 4 shown in FIGS. 11 to 14 are used as basic cases, and other cases are used as modified cases, and processing is performed so as to return to these basic cases.
[0085]
FIG. 15 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 5, and (B) a flowchart illustrating processing contents.
[0086]
Case 5 is a case where the subject is held in a vertical position where the right hand holding the digital camera 1 is up and the subject is photographed in a plurality of pieces in order from the bottom to the top. At this time, since shooting is performed in the right direction on the long side of the shooting screen, the first byte of the identification code is 1, and the second byte of the identification code is in ascending order from bottom to top.
[0087]
Therefore, in this case 5, as shown in FIG. 15B, first, the splicing process shown in FIG. 9 is performed (step S59), and then the result image is rotated 90 degrees counterclockwise (step S59). S60), return. Here, the process of step S59 is the same as the content of the case 1 process, which is one of the basic cases, and the process of step S60 is performed by the user who viewed the result image.
[0088]
FIG. 16 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 6, and (B) a flowchart illustrating processing contents.
[0089]
This case 6 is a case where the subject is held in a vertical position so that the right hand holding the digital camera 1 is up, and the subject is photographed by dividing it into a plurality of sheets in order from the upper side to the lower side. At this time, since shooting is performed in the left direction on the long side of the shooting screen, the first byte of the identification code is 2, and the second byte of the identification code is in ascending order from top to bottom.
[0090]
Therefore, in this case 6, as shown in FIG. 16B, the image data is first rearranged so that the second byte of the identification information is in descending order (step S61), and the splicing process shown in FIG. (Step S62), and then the resulting image is rotated 90 degrees counterclockwise (step S63) and the process returns. Here, the processing of step S61 and step S62 is the same as the content of the case 2 processing which is one of the basic cases, and the processing of step S63 is performed by the user who has seen the result image.
[0091]
FIG. 17 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 7, and (B) a flowchart illustrating processing contents.
[0092]
This case 7 is a case where the subject is held in a vertical position with the right hand holding the digital camera 1 at the bottom and the subject is photographed in a plurality of pieces in order from the bottom to the top. At this time, since shooting is performed in the left direction on the long side of the shooting screen, the first byte of the identification code is 2, and the second byte of the identification code is in ascending order from bottom to top.
[0093]
Accordingly, in this case 7, as shown in FIG. 17B, the image data is first rearranged so that the second byte of the identification information is in descending order (step S64), and the splicing process shown in FIG. 9 is performed. (Step S65), and then the resulting image is rotated 90 degrees clockwise (step S66) and the process returns. Here, the processing of step S64 and step S65 is the same as the content of the case 2 processing, which is one of the basic cases, and the processing of step S66 is performed by the user who viewed the result image.
[0094]
FIG. 18 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 8, and (B) a flowchart illustrating processing contents.
[0095]
This case 8 is a case where the subject is held in a vertical position with the right hand holding the digital camera 1 at the bottom, and the subject is photographed by dividing it into a plurality of sheets in order from the upper side to the lower side. At this time, since shooting is performed in the right direction on the long side of the shooting screen, the first byte of the identification code is 1, and the second byte of the identification code is in ascending order from top to bottom.
[0096]
Accordingly, in this case 8, as shown in FIG. 18B, the splicing process shown in FIG. 9 is first performed (step S67), and then the result image is rotated 90 degrees clockwise (step S68). ), Return. Here, the process of step S67 is the same as the content of the case 1 process, which is one of the basic cases, and the process of step S68 is performed by the user who viewed the result image.
[0097]
FIG. 19 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 9, and (B) a flowchart illustrating processing contents.
[0098]
This case 9 is a case where the subject is held in a horizontal position where the right hand holding the digital camera 1 is on the right and the subject is photographed in a plurality of pieces in order from the lower side to the upper side. At this time, since shooting is performed in the upward direction on the short side on the shooting screen, the first byte of the identification code is 3, and the second byte of the identification code is in ascending order from bottom to top.
[0099]
Accordingly, in this case 9, as shown in FIG. 19B, all the processing target images are first rotated 90 degrees clockwise (step S69), and the splicing process shown in FIG. 9 is performed (step S70). Then, the result image is rotated 90 degrees counterclockwise (step S71), and the process returns. Here, the processing of step S69 and step S70 is the same as the content of the case 3 processing, which is one of the basic cases, and the processing of step S71 is performed by the user who viewed the result image.
[0100]
FIG. 20 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 10, and (B) a flowchart illustrating processing contents.
[0101]
This case 10 is a case where the subject is held in a horizontal position where the right hand holding the digital camera 1 is on the right and the subject is photographed in a plurality of pieces in order from the upper side to the lower side. At this time, since shooting is performed in the downward direction on the short side on the shooting screen, the first byte of the identification code is 4, and the second byte of the identification code is in ascending order from top to bottom.
[0102]
Therefore, in this case 10, as shown in FIG. 20B, all the processing target images are first rotated 90 degrees clockwise (step S72), and the image data is set so that the second byte of the identification information is in descending order. Are rearranged (step S73), the splicing process shown in FIG. 9 is performed (step S74), and then the result image is rotated 90 degrees counterclockwise (step S75) and the process returns. Here, the processing from step S72 to step S74 is the same as the content of the case 4 processing which is one of the basic cases, and the processing of step S75 is performed by the user who has seen the result image.
[0103]
FIG. 21 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 11, and (B) a flowchart illustrating processing contents.
[0104]
This case 11 is a case where the subject is held in a vertical position where the right hand holding the digital camera 1 is up and the subject is photographed in a plurality of pieces in order from the left side to the right side. At this time, since shooting is performed in the downward direction on the short side on the shooting screen, the first byte of the identification code is 4, and the second byte of the identification code is in ascending order from left to right.
[0105]
Therefore, in this case 11, as shown in FIG. 21B, all the processing target images are first rotated 90 degrees clockwise (step S76), and the image data is set so that the second byte of the identification information is in descending order. Are rearranged (step S77), the splicing process shown in FIG. 9 is performed (step S78), and then the result image is rotated 180 degrees (step S79), and the process returns. Here, the processing from step S76 to step S78 is the same as the content of the case 4 processing which is one of the basic cases, and the processing of step S79 is performed by the user who has seen the result image.
[0106]
FIG. 22 is a diagram illustrating (A) the direction and order of captured images together with an identification code in case 12, and (B) a flowchart illustrating processing contents.
[0107]
This case 12 is a case where the subject is held in a vertical position where the right hand holding the digital camera 1 is up and the subject is photographed in a plurality of pieces in order from the right side to the left side. At this time, since shooting is performed upward on the short side on the shooting screen, the first byte of the identification code is 3, and the second byte of the identification code is in ascending order from right to left.
[0108]
Therefore, in this case 12, as shown in FIG. 22B, all the processing target images are first rotated 90 degrees clockwise (step S80), and the splicing process shown in FIG. 9 is performed (step S80). Thereafter, the result image is rotated by 180 degrees (step S82), and the process returns. Here, the processing of step S80 and step S81 is the same as the content of the case 3 processing which is one of the basic cases, and the processing of step S82 is performed by the user who has seen the result image.
[0109]
As described above, the state at the time of photographing is classified into twelve, but it can be seen that all four cases are reduced except for the processing portion by the user's operation. Which basic case is applied depends on the value of the first byte of the identification code, that is, processing of case 1 if the first byte is 1, processing of case 2 if the first byte is 3, If there is, processing in case 3 and processing in case 4 are automatically performed.
[0110]
Further, in any of these four basic cases, since the splicing process of FIG. 9 is performed by the same algorithm, the processing program is extremely compact and efficient.
[0111]
In the above description, a part of each case processing is performed by a user's operation. However, a gravitational sensor or the like is provided in the digital camera 1 to detect the shooting posture of the camera, or an angular velocity sensor or the like is added to the digital camera 1. If the rotation angle and the rotation direction are detected and the detected information is added as the header information of the image file, it can be automatically performed without any user operation.
[0112]
Next, FIG. 23 is a flowchart showing the contents of the manual processing in step S8 of FIG. 4 mainly by the state of the display screen.
[0113]
The result image 32 stitched by the automatic processing as described above is displayed in the screen 31 in step S6 of FIG. The screen 31 further includes a joint OK button 33 for confirming the result image 32 when the user who has seen the result image 32 is satisfied with the result, and manual reprocessing without the user being satisfied. A manual button 34 to be operated when performing and a process end button 35 for exiting this process are displayed so as to be operable by the mouse 6, the keyboard 5, or the like.
[0114]
When the manual button 34 is operated in this state (step S91), first the first image (image 1) 36 and the second image are manually joined in order as shown in FIG. An image (image 2) 37 is displayed in the screen 31. Further, on the screen 31 at this time, a message 38 for prompting the user to designate the alignment indexes 36a and 37a for indicating the matching portions of the first image 36 and the second image 37 is displayed. In addition, the screen 31 displays the process end button 35 and an execution button 39 for proceeding to the next process.
[0115]
When the execution button 39 is operated after designating the alignment indexes 36a and 37a (step S92), the CPU 11 performs a manual joining process so that the alignment indexes 36a and 37a match (step S93). ).
[0116]
Then, the result image 40 obtained by performing the splicing is displayed on the screen 31. If there is image data to be spliced subsequently (for example, the third image 41 in the figure), this is also indicated. indicate. At this time, when the image becomes long and cannot be displayed at the same time, a slide bar 42 that can change the display position by a slide operation is displayed. In addition, on this screen 31, the seaming OK button 33 and the process end button 35 are displayed, and the user does not like the result of seaming the first image 36 and the second image 37 manually. A return button 43 is displayed to be operated when it is desired to perform splicing again.
[0117]
When the return button 43 is operated (step S94), the process can return to the step S92 and the manual joining process can be performed again.
[0118]
According to such an embodiment, since the identification information is provided as the header information of the image file, it is possible to determine which image is a target to be spliced. A plurality of processes prepared for each case can be selected on the basis of the information on the photographing order, and the seaming process can be automatically performed.
[0119]
In addition, multiple cases corresponding to the shooting direction and shooting order are classified into some basic processing cases and other processing cases, and the basic processing cases remain as part of other processing cases. Since it can be applied, the image composition processing program can be reduced in weight and speed, and the capacity occupied by the main memory, the internal recording medium, etc. can be reduced.
[0120]
Further, since all the basic algorithms for the splicing process are the same, it is possible to further reduce the weight and speed of the image composition processing program.
[0121]
Also, when programming the image composition processing program, the time for coding and debugging can be shortened, and the development efficiency is improved.
[0122]
Note that the present invention is not limited to the above-described embodiments, and various modifications and applications can be made without departing from the spirit of the invention.
[0123]
【The invention's effect】
As described above, according to the image composition processing apparatus of the present invention according to claim 1, since a plurality of pieces of image information are rearranged on the basis of the accompanying information and then merged together, the efficient processing is performed. be able to.
[0125]
  And claims2According to the image composition processing apparatus of the present invention according to claim,1In addition to the effects similar to those of the invention described above, the other image composition processing cases are reduced to the four basic image composition processing cases, so that more efficient processing can be performed.
[0126]
  Claim3According to the image composition processing apparatus of the present invention according to claim,2It is possible to easily determine which of the four basic image composition processing cases should result from the information related to the shooting direction in the accompanying information.
[0127]
  Claim4According to the image composition processing apparatus of the present invention according to claim,3As described above, the four basic image composition processing cases include a predetermined algorithm and perform separate processing in other portions, thereby achieving efficient processing. It can be carried out.
[0128]
  Claim5According to the image composition processing apparatus of the present invention according to claim,4In addition to the same effects as the invention described in, the processing load is reduced by enabling manual rotation operations for the other eight image composition processing cases other than the four basic image composition processing cases. be able to.
  According to the image composition processing apparatus of the present invention according to claim 6, the same effect as that of the invention according to claim 4 can be obtained, and automation can be performed without being operated by a user.
[0129]
According to the image composition processing apparatus of the present invention according to claim 7, the same effect as that of the invention according to claim 1 can be obtained, and more efficient processing can be performed by rotating a plurality of pieces of image information. Can do.
[0130]
According to the image composition processing apparatus of the present invention according to claim 8, the same effect as that of the invention according to claim 1 can be obtained, and more efficient processing can be performed by including a part that is the same algorithm. it can.
[0131]
According to the image composition processing apparatus of the present invention according to claim 9, the same effect as that of the invention according to claim 8 is achieved, and efficient processing using the same algorithm is possible by sequentially joining image information. It becomes.
[0132]
According to the recording medium on which the image composition processing program of the present invention is recorded according to claim 10, the computer can rearrange a plurality of pieces of image information on the basis of the accompanying information and then combine them together to synthesize the efficient processing. Can be performed.
[0133]
According to the recording medium on which the image composition processing program of the present invention according to claim 11 is recorded, the computer efficiently rotates a plurality of pieces of image information after rotating and rearranging them based on the accompanying information. Can be processed.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of a system to which an image composition processing apparatus according to an embodiment of the present invention is applied.
FIG. 2 is a block diagram showing an internal configuration of the personal computer according to the embodiment.
FIG. 3 is a view showing the structure of data recorded on the recording medium of the embodiment.
FIG. 4 is a flowchart showing a main routine of an image composition processing program executed by the personal computer of the embodiment.
FIG. 5 is a flowchart showing details of display determination processing in the embodiment.
FIG. 6 is a diagram showing an example of changing the display color of a frame of an image to be processed in the embodiment.
FIG. 7 is a diagram showing an example of displaying a mark on a frame of an image to be processed in the embodiment.
FIG. 8 is a flowchart showing target image automatic detection processing in the embodiment.
FIG. 9 is a flowchart showing a basic algorithm of splicing processing in the embodiment.
FIG. 10 is a diagram showing a state of images stitched together by the algorithm shown in FIG. 9;
11A is a view showing the direction and order of captured images together with an identification code in case 1 processing of the embodiment, and FIG. 11B is a flowchart showing processing contents;
12A is a view showing the direction and order of captured images together with an identification code in case 2 processing of the embodiment, and FIG. 12B is a flowchart showing processing contents;
13A is a view showing the direction and order of captured images together with an identification code in case 3 processing of the embodiment, and FIG. 13B is a flowchart showing processing contents;
14A is a view showing the direction and order of captured images together with an identification code in case 4 processing of the embodiment, and FIG. 14B is a flowchart showing processing contents;
15A is a view showing the direction and order of captured images together with an identification code in case 5 processing of the embodiment, and FIG. 15B is a flowchart showing processing contents;
16A is a view showing the direction and order of captured images together with an identification code in case 6 processing of the embodiment, and FIG. 16B is a flowchart showing processing contents;
FIGS. 17A and 17B are diagrams showing (A) the direction and order of captured images together with an identification code, and (B) a flowchart showing the processing contents in case 7 processing of the embodiment.
18A is a view showing the direction and order of captured images together with an identification code in case 8 processing of the embodiment, and FIG. 18B is a flowchart showing processing contents.
19A is a view showing the direction and order of captured images together with an identification code in case 9 processing of the embodiment, and FIG. 19B is a flowchart showing processing contents;
20A is a view showing the direction and order of captured images together with an identification code in case 10 processing of the embodiment, and FIG. 20B is a flowchart showing processing contents.
FIGS. 21A and 21B are diagrams showing (A) the direction and order of captured images together with an identification code, and (B) a flowchart showing the processing contents in case 11 processing of the embodiment.
22A is a view showing the direction and order of captured images together with an identification code in case 12 processing of the embodiment, and FIG. 22B is a flowchart showing processing contents;
FIG. 23 is a flowchart showing the contents of manual processing in the embodiment mainly by the state of a display screen.
[Explanation of symbols]
1 ... Digital camera
2 ... Recording medium
3. Personal computer
11 ... CPU (rearrangement means, image composition means)
12. Main memory (recording medium on which an image composition processing program is recorded)
13: Internal recording medium (recording medium on which an image composition processing program is recorded)

Claims (11)

A plurality of image information captured with a mutually overlapping portion min in order to constitute an object image, among the two directions of the two directions and the short side of the long sides of the photographing screen comprising a rectangular these image information A recording medium capable of recording a shooting direction indicating one direction and accompanying information including information related to the shooting order of the plurality of pieces of image information ;
Rearrangement means for rearranging the plurality of pieces of image information in ascending order or descending order of the photographing order based on information relating to the photographing direction of the accompanying information;
Image combining means for combining the image information rearranged in the ascending order or descending order by the rearranging means from one direction so as to become one image; and
An image composition processing apparatus comprising: The image synthesizing means has four basic image synthesizing processing cases in the two directions related to the long side and the two directions related to the short side of the rectangular imaging screen. 2. The image composition processing apparatus according to claim 1 , wherein the image composition processing apparatus is reduced to any one of four basic image composition processing cases. The image composition processing case possessed by the image composition means includes one case where the long side of the shooting screen is in the horizontal position during shooting and two cases where the long side of the shooting screen is in the vertical position during shooting in the four directions. By having each of them, a total of 12 image composition processing cases are provided, and the other eight image composition processing cases excluding the four basic image composition processing cases have information on the shooting direction in the accompanying information. 3. The image composition processing apparatus according to claim 2 , wherein each of the four basic image composition processing cases is the same, and the image composition processing apparatus is reduced to any one of the four basic image composition processing cases. In the above four basic image composition processing cases, the first basic image composition is made by sequentially joining together a plurality of pieces of image information photographed in the first direction among the four directions by a predetermined algorithm. A processing case and a plurality of pieces of image information photographed in a second direction opposite to the first direction are rearranged in descending order of the photographing order, and then sequentially joined by the predetermined algorithm to form a composite image. The basic image composition processing case and a plurality of pieces of image information photographed in a third direction orthogonal to the first direction are each spliced by 90 degrees in one rotation direction and then sequentially spliced by the predetermined algorithm. A third basic image composition processing case to be a composite image, and a plurality of pieces of image information photographed in a fourth direction opposite to the third direction. Same as case And a fourth basic image composition processing case in which the images are rearranged in descending order of the photographing order after being rotated 90 degrees in the rotation direction, and then sequentially joined by the predetermined algorithm to form a composite image. The image composition processing apparatus according to claim 3 . The other eight image composition processing cases other than the four basic image composition processing cases are performed manually after performing any one of the first to fourth basic image composition processing cases. The image composition processing apparatus according to claim 4 , wherein a rotation operation is possible. The accompanying information further includes information related to the shooting posture of the camera,
  The image synthesizing means, for the other eight image synthesis processing cases other than the four basic image synthesis processing cases, based on the information related to the shooting posture of the camera in the accompanying information, The image composition processing device according to claim 4, wherein the image composition processing device is rotated. 2. The image composition according to claim 1, wherein the rearrangement unit has a function of rearranging the plurality of pieces of image information in ascending or descending order of the photographing order and rotating the plurality of pieces of image information. Processing equipment.   2. The image composition processing apparatus according to claim 1, wherein the image composition processing by the image composition means includes a portion having the same algorithm regardless of the photographing direction and the photographing order of the plurality of pieces of image information.   The above algorithm joins one image information and other image information to form a composite image, and when there is other image information to be spliced, the composite image is spliced. The image composition processing apparatus according to claim 8, wherein the image composition processing apparatus is provided. A computer, a recording medium recording an image synthesizing program for causing the process of splicing a plurality of image information captured with a mutually overlapping portion min in order to constitute an object image, the image synthesizing program On the computer
The accompanying information including information on the shooting direction indicating one of the two directions related to the long side and the two directions related to the short side of the shooting screen formed of a rectangle of the image information and the shooting order of the plurality of image information , Based on the information related to the shooting direction, the plurality of image information is rearranged in ascending order or descending order of the shooting order ,
The image information rearranged in ascending or descending order is synthesized by joining from one direction so as to become one image,
A recording medium on which an image composition processing program is recorded. A computer, a recording medium recording an image synthesizing program for causing the process of splicing a plurality of image information captured with a mutually overlapping portion min in order to constitute an object image, the image synthesizing program On the computer
The accompanying information including information on the shooting direction indicating one of the two directions related to the long side and the two directions related to the short side of the shooting screen formed of a rectangle of the image information and the shooting order of the plurality of image information , based on the information relating to the photographing direction, Rutotomoni rotate on SL plurality of image information, the plurality of image information is rearranged in ascending or descending order of the photographing order,
Characterized in that said rotation is image information rearranged in the ascending or descending order, in which is synthesized by joining the one direction so that the first image, a recording medium recording an image synthesizing program.

Images