JP2000358194A - Image compositing device and method and storage medium storing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily execute an image compositing process regardless of the capacity of a data buffer even if two images of superhigh accuracy are processed to each other when the images are composited between the image files of very large capacities compared with a system mounted memory. SOLUTION: A data management part 102 produces reduced images of each of image data which are stored in a file store part 110 and to be composited together and stores these reduced images in a data buffer 103. Then the part 102 extracts the overlapping area of every reduced image and performs temporary alignment of the reduced image according to the extracted overlapping area to calculate the information on the overlapping area and its peripheral areas. An image processing part 101 composition accurately the image data according to the calculated information and produces a single sheet of images.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing apparatus, a method and a program which are suitable for use in synthesizing very large image files, especially in mosaic synthesizing between ultra-high definition images. And a storage medium storing the information.

[0002]

2. Description of the Related Art An example of a conventional image synthesizing apparatus is a software product attached to a digital camera product.
Software such as photoStitch (Canon Inc.) is known. Also, Japanese Patent Application Laid-Open No. 9-32205
No. 9 discloses an apparatus that creates a single image by performing panoramic synthesis on a plurality of images having overlapping portions, that is, a device that performs a mosaic synthesis process in the image processing field.

As shown in FIG. 8, this conventional image synthesizing apparatus mainly comprises application software, an operating system, and hardware. The conventional image synthesizing apparatus having such a configuration operates as follows.

In the actual processing of image synthesis, first, image data is read from a camera or a file system into an image data management system section in application software, and is subjected to synthesis based on attribute information attached to the read image data. Specify image data. A combining process is performed on the specified plurality of image data. As processing for combination, first, a superposition position is designated manually or automatically, then corresponding point extraction processing and combination parameter calculation are performed, and actual combination processing is performed.

[0005] In the above-mentioned publication, the flow of the synthesizing process for two images is shown in FIG. 9. That is, a set of images to be combined is read into the application software, and the combined image area is set to 2 of the image to be combined.
The size is doubled, and the image to be combined is sequentially copied to the combined image area while performing coordinate conversion.
When the number of images to be combined increases, the image to be combined and the combined image area having the total size thereof are similarly secured in the application software and combined.

[0006]

However, the conventional example has the following problems. Since all image areas are secured in the application software, if the size of one input image is very large, or if the number of images to be combined is large, the amount of image data to be processed However, in some cases, the size of the memory becomes larger than the memory that can be secured by the application software included in the processing device, and the processing time may be extremely long or the processing may not be possible.

An object of the present invention is also to be applied to a case where the synthesized image is very large, such as when the size of one input image to be synthesized is large or when the number of input images is large. Another object of the present invention is to provide a system capable of accurately synthesizing images regardless of the amount of memory included in the processing device.

[0008]

In order to achieve the above object, an image synthesizing apparatus according to the present invention comprises a generating means for generating each reduced image of a plurality of image data to be image-combined; Extracting means for extracting an overlapping area of the image; calculating means for calculating information necessary for positioning of each image data by performing positioning of each reduced image based on the extracted overlapping area; and A synthesizing unit for synthesizing each image data based on the information to create one image.

Further, the image synthesizing method according to the present invention comprises a creating step of creating respective reduced images of a plurality of image data to be combined, an extracting step of extracting an overlapping area of the created reduced images, and an extracting step. Calculating the information required for the alignment of each image data by performing the alignment of each reduced image based on the overlapped region thus obtained; and synthesizing each image data based on the calculated information to obtain 1 And a synthesizing step of creating one image.

[0010] The storage medium storing the program according to the present invention includes a procedure for creating each reduced image of a plurality of image data to be combined with an image, a procedure for extracting an overlapping area of each created reduced image, A procedure for calculating information necessary for alignment of each image data by performing alignment of each reduced image based on the extracted overlapping area, and combining each image data based on the calculated information to obtain 1 And a program for executing a procedure for creating a single image.

The synthesizing means may generate a reduced image when the total data amount of the plurality of image data exceeds a predetermined amount. A reduced image may be created by selecting a reduced ratio.

Further, the calculating means may calculate information indicating an overlapping area of the extracted reduced image and a peripheral area thereof.

[0013]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 shows the first embodiment of the present invention.
1 is a block diagram illustrating an image synthesizing apparatus according to an embodiment.

In FIG. 1, the image synthesizing apparatus includes a computer (central processing unit; processor; data processing unit) 100 which operates under program control, and a file storage unit 110. The computer 100 includes an image processing unit 101, a data management unit 102, and a data buffer 103.

Each of the above-mentioned parts operates as follows. The image processing unit 101 performs alignment processing for image synthesis on an image read into the data buffer 103, calculates parameters for alignment calculation,
Then, a pixel combining process is performed using the calculated parameters.

The data management unit 102 includes a file storage unit 1
Data buffer 10 for image data stored in 10
3 and a thinning-out reduction process.

The data buffer 103 includes the image processing unit 10
A data area that can be accessed at high speed from 1 and stores an image to be processed by the image processing unit 101.

Next, the overall operation of the present embodiment will be described in detail with reference to the flowcharts of FIGS. First, a user designates individual file names or a group of images to be combined, that is, a set of image files to be combined, or collectively describes them in a setting file and designates them. The data management unit 102 registers the specified image name in the data management unit. The images to be combined are image data in which some of the peripheral portions of the images overlap each other (step A1 in FIG. 2).

Next, the data management unit 102 reads the header information of each image stored in the file storage unit 110, calculates the amount of memory required when each image is expanded on the memory, and If the sum is larger than a threshold value indicating the amount of image data that can be processed at one time, which is calculated in advance based on the size of the data buffer, the pixel is set so that the total image data amount becomes smaller than the threshold value. Are thinned out at regular intervals, or the average value is obtained, so that the smaller, for example, the height and width are 1/8 to 1/1, respectively.
A reduced image having an image size of 6 is generated and stored.

At this time, the correspondence between the original image file name and the file name of the reduced image, the reduction ratio, and the like are stored as management information in the data management unit 102 (step A2).
The set of reduced images generated in this way has a size that can be collectively read into the data buffer 103.

Next, the set of reduced images is stored in the data buffer 10.
3, and all of the reduced images are manually or automatically aligned. In this step, by listing and combining all the images, it is possible to easily confirm which part of each image overlaps. For example, in the case of images A and B as shown in FIG. 3, the alignment processing between reduced images is performed by manually designating corresponding points as shown in the above-mentioned prior art publication by an instruction means, or by automatically specifying the corresponding points. This can be done by finding the correspondence between characteristic points (step A3).

Assuming that the displacement of each image set can be expressed by translation, rotation, and enlargement between the images, a parameter for synthesis is calculated by using information of a plurality of corresponding points, for example, using the least square method. can do. By applying the transformation with this parameter to the four corner points of each image,
For example, an overlap area between the reduced images as shown in FIG. 4A can be calculated (step A4).

Further, by using the management information created when the reduced image is first created, the overlapping area between the reduced images in FIG. 4A is changed to the original image to be synthesized as shown in FIG. It is possible to calculate which range the file corresponds to (step A5).

Next, a detailed registration process is sequentially performed on the overlapping areas included in the original image file set to be combined. In the alignment processing performed on the reduced image, the accuracy used is low because the image used for alignment is smaller than the original image to be combined. Therefore, step A5
The overlapping area and its surrounding area of FIG. 4B calculated in step (b) are sequentially read into the data buffer 103, and accurate positioning is performed. In the registration process, the positional relationship of the entire image is
Since it is known at the time of the alignment in the reduced image, there is no problem even in the manual alignment, even if the alignment is performed only by the detailed features in the partial image, not by the relation of the whole image. This alignment processing can be performed by manually specifying corresponding points as in step A3, or by automatically associating feature points (step A6).

Further, parameters for positioning are calculated from the accurate positioning processing performed in step A6, similarly to step A4 (step A7). Finally, the area for creating the entire composite image is stored in the file storage unit 110.
The composite image can be finally created by copying the image data from the compositing target image file on the file storage unit 110 to the composite image file using the parameters calculated in step A7. (Step A8).

Next, the calculation of the overlapping area and its surrounding area will be described in detail. In FIG. 5, (b) shows a case where two images A and B are overlapped so that the corresponding points 501 in (a) coincide. First, using the conversion function F obtained from the corresponding points, the points (0, 0), (x0,
0), (0, y0), and (x0, y0) in the coordinate system of the image A are obtained.

Next, an area included in the image A among these areas is obtained. Further, in the coordinate system of the image A, a region including the determined region and having sides parallel to the x-axis and the y-axis is determined.
This is an overlapping area. By dividing the coordinate value of the overlapping area in the reduced image by the reduction ratio managed by the data management unit 102, the coordinate value representing the boundary of the overlapping area in the original high-definition image can be obtained.

Finally, an area having a constant width around the overlapping area is considered as shown in FIG. 5B, and this area is defined as a peripheral area. The overlapping area and the peripheral area in the image B can be similarly obtained.

According to the present embodiment, the size of the image file to be synthesized is checked in advance, a reduced image having a size that can be accommodated in the data buffer 103 is created in advance, the registration process is performed using the reduced image, and the Since only the area and its surrounding area are sequentially read and the detailed registration and combination processing are performed, when the size of one input image to be combined is large or when the number of input images is large For example, even in the case where the synthesized image is a very large image, there is an effect that image synthesis can be performed regardless of the amount of memory included in the processing device.

Next, a second embodiment of the present invention will be described. In the present embodiment, as shown in FIG. 6, a computer (central processing unit; processor; data processing unit) 100 operated by program control and a file storage unit 110 are provided.

The computer 100 includes an image processing unit 101
And a hierarchical data management unit 104 and a data buffer 103. The operations of these units are the same as in the first embodiment except for the hierarchical data management unit 104.

The hierarchical data management unit 104 operates as follows. When the image data is designated, a hierarchical reduced image of the image data is created, and the hierarchical information is managed.
In addition, an appropriate partial image is read into the data buffer as needed.

Next, the overall operation of the present embodiment will be described with reference to the flowcharts of FIGS. First, the user designates a set of images to be combined, that is, a set of image files to be combined, by specifying individual file names or by describing them collectively in a setting file. The hierarchical data management unit 102 registers the specified image name in the hierarchical data management unit. The images to be combined are image data in which some of the peripheral portions of the images overlap each other (step B1 in FIG. 7).

Next, the hierarchical data management unit 102 reads the header information of each image stored in the file storage unit 110, calculates the amount of memory required when each image is expanded on the memory, and calculates each image data. Is larger than a threshold value indicating the amount of image data that can be processed at one time, which is calculated in advance based on the size of the data buffer, and creates an image in which each image is reduced by half in both the vertical and horizontal directions. The reduced image is stored in the file storage unit 110. further,
If the image data amount of the reduced image still exceeds the threshold value, the image is further reduced to half length and width. This is repeated until the amount of image data falls below the threshold. At this time, the correspondence between the original image file name and the file name of the hierarchical reduced image is stored as management information in the data management unit (step B2).

The set of the lowest-order reduced images generated in this way has a size that can be collectively read into the data buffer 103. Next, the lowest-order reduced image set is entirely read into the data buffer 103, and the reduced images are aligned manually or automatically. In this step, by listing all the images and combining them, it is possible to easily confirm which part of each image overlaps. The alignment processing between the area reduced images and the calculation of the alignment parameter and the overlapping area can be performed manually or automatically as in the case of the first embodiment (step B3, step B4).

Further, by using the management information created when the reduced image is first created, an overlapping area between images having a size one order higher than each reduced image, that is, two times vertically and horizontally, is calculated. (Step B5).

Next, a detailed positioning process is sequentially performed on the overlapping area included in the reduced image of the next higher hierarchy. In the positioning process performed on the lower reduced image, the accuracy is low because the image used for positioning is small. For this reason, the overlapping area and its surrounding area calculated in step B5 are sequentially read into the data buffer 103, and accurate positioning is performed. This alignment processing can be performed by manually specifying corresponding points as in step B3, or by automatically associating feature points. Compared to the case where the parameters are obtained with the reduced image reduced to 1/8 to 1/16 in the first embodiment and the overlap region is obtained, the range of the error can be assumed to be narrower, and the width of the peripheral region is smaller. Can be smaller. (Step B
6).

In the detailed registration process, since the range is limited by the registration process between reduced images, it is not necessary to read the entire original image file to be combined into the data buffer. In addition, since the positional relationship of the entire image is known at the stage of positioning in the reduced image, even in manual positioning, positioning is performed based on only detailed features in the image, not the entire image. There is no problem.

Further, parameters for positioning are calculated from the detailed positioning processing performed in step B6, as in step B4 (step B7).

The alignment processing in steps B6 and B7 is sequentially performed until the uppermost layer, that is, the alignment of the original high-definition image is performed.

Lastly, an area for creating the entire composite image is secured in the file storage unit 110, and the parameters calculated in step B7 are used to convert the image file to be synthesized in the file storage unit 110 into a composite image file. By sequentially copying the image data, a composite image can be finally created (step B8).

According to the present embodiment, the effect of the first embodiment can be obtained, and a hierarchical reduced image is created in advance and the positioning process is performed step by step. The capacity of 103 can always be used efficiently.

When the first and second embodiments are configured using a computer 100, a memory such as a ROM used in the computer system constitutes a storage medium according to the present invention. In this storage medium, a program for executing the procedure for performing the operation described in the flowcharts of FIGS. 2 and 7 is stored. As the storage medium, a semiconductor storage device, an optical disk, a magneto-optical disk, a magnetic recording medium, or the like is used.

[0044]

As described above, according to the present invention,
By creating reduced images of a plurality of input images to be combined in advance, and temporarily aligning them, information necessary for the image data combining process is obtained, and the normal position of the image data is determined using this information. Since the synthesizing process is performed by performing the matching, the memory capacity required for the synthesizing process can be reduced.

In particular, the processing apparatus is provided even in the case where the size of one input image to be synthesized is large or the number of input images is large, such as when the synthesized image is very large. It is possible to provide a system capable of synthesizing images regardless of the amount of memory.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 2 is a flowchart showing an operation of the first embodiment.

FIG. 3 is a configuration diagram illustrating an example of a screen for designating a corresponding point according to the first embodiment;

FIG. 4 is a configuration diagram illustrating a relationship between overlapping areas in the reduced image and the original high-definition image according to the first embodiment.

FIG. 5 is a configuration diagram illustrating an overlapping area and a peripheral area according to the first embodiment;

FIG. 6 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

FIG. 7 is a flowchart illustrating the operation of the second embodiment.

FIG. 8 is a block diagram showing a system configuration of a conventional image synthesizing apparatus.

FIG. 9 is a flowchart showing the operation of a conventional image synthesizing apparatus.

[Explanation of symbols]

REFERENCE SIGNS LIST 100 computer (central processing unit; processor; data processing unit) 101 image processing unit 102 data management unit 103 data buffer 104 hierarchical data management unit 110 file storage unit

Claims (6)

[Claims] A generating unit configured to generate each reduced image of a plurality of image data to be image-combined; an extracting unit configured to extract an overlapping area of each of the generated reduced images; Calculating means for calculating information necessary for the alignment of each image data by performing the alignment of each reduced image, and synthesizing each image data based on the calculated information to create one image An image synthesizing apparatus, comprising: synthesizing means for performing image synthesis. 2. The image synthesizing apparatus according to claim 1, wherein said synthesizing means generates a reduced image when a total data amount of said plurality of image data exceeds a predetermined amount. 3. The image synthesizing apparatus according to claim 2, wherein said synthesizing means selects a reduction ratio at which a total data amount of each reduced image is equal to or less than the predetermined amount to generate a reduced image. 4. The apparatus according to claim 1, wherein said calculating means calculates information indicating an overlapping area of the extracted reduced image and a peripheral area thereof. 5. A creating step of creating each reduced image of a plurality of image data to be image-synthesized, an extracting step of extracting an overlapping area of each of the created reduced images, and A calculation step of calculating information necessary for alignment of each image data by performing alignment of each reduced image by using the above-mentioned information; and creating one image by synthesizing each image data based on the calculated information. And a synthesizing step. 6. A procedure for creating each reduced image of a plurality of image data to be image-synthesized, a procedure for extracting an overlapping area of each of the created reduced images, A procedure of calculating information necessary for the alignment of each image data by performing the alignment of the reduced images; and a procedure of creating one image by combining each image data based on the calculated information. Storage medium storing a program for executing the program.