JP2000322563A - Device and method for preparing panoramic image and computer readable storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a processing time in the case of preparing a panoramic image by using a plurality of compressed picture data. SOLUTION: Four pieces of compressed picture data are successively stored in memories 1 to 4 through an inputting part 10 and a switch 20. A switch 70 successively sends image data of each memory to a decoding part 150 and a DC image preparing part 80. The part 80 respectively prepares a DC image by using a DC component of each block of every image data. A switch 90 successively stores respective DC images by two at a time in DC image memories 100 and 110. A relative position detecting means 120 detects the relative position of two DC images, and an image selecting part 30 selects plural images needed for panoramic synthesis from the detected relative position. An image synthesizing part 160 receives decoded image data of the plural selected images from a decoding part 150 and prepares a panoramic image by synthesizing the image data on the basis of the relative position data stored in a relative position data part 140.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a panoramic image forming apparatus and method suitable for forming a panoramic image by connecting a plurality of image data compressed by using an orthogonal transform, and a computer used for the method. It relates to a readable storage medium.

[0002]

2. Description of the Related Art FIG. 5 schematically shows a conventional general panoramic image creation process. In FIG. 5, (a) shows 3
These are images. These three images have portions that partially overlap as shown in FIG. The process of combining these images to create a single image as in (c) is called a panoramic image combining process.

[0003] Such a panoramic image synthesizing process is performed when an image with a high resolution or an image with a wide field of view is desired. For example, a digital camera or software for a personal computer attached to the digital camera is used. But it has been done. In this process, a process of comparing image data of a plurality of images to detect a relative position and synthesizing them so that they are overlapped is performed.

Generally, image data is often compressed and stored. As a compression method, compression using orthogonal transform is most often used.
T conversion is JPEG that handles still images, MPEG that handles moving images,
Also used in DV format. Actually, one screen is divided into small blocks of N × N pixels, and DCT transform is performed on each block.

In a panoramic image synthesizing process, if a plurality of image data are compressed, each image data is decoded, and then the decoded image data is compared to detect a relative position and synthesize them. Processing is performed.

[0006]

However, in the conventional panoramic image synthesizing method, when a plurality of image data are compressed, each image data is decoded, and then the decoded image data is compared. Therefore, there is a problem that decoding processing and image data comparison processing take time.

When a panorama synthesizing process is performed when there are a plurality of continuous image data at very short intervals such as moving image data, each image data is also compared to compare the preceding and succeeding images. There is a problem that decoding is required and the processing time is further increased.

The present invention has been made to solve the above-mentioned problem, and has as its object to reduce the processing time of the panorama synthesizing process.

[0009]

In order to achieve the above object, a panoramic image creating apparatus according to the present invention comprises:
An input unit for inputting a plurality of image data, a DC image generating unit for extracting a DC component from each of the input plurality of image data to generate a plurality of DC images, and comparing the generated plurality of DC images Comparing means, selecting means for selecting a plurality of image data required for panoramic synthesis from the plurality of image data based on a comparison result of the comparing means, decoding means for decoding the selected plurality of image data, A synthesizing means for synthesizing the plurality of decoded image data based on the comparison result of the detected comparing means to create a panoramic image.

In the panoramic image creating method according to the present invention, a procedure for inputting a plurality of image data, a procedure for extracting a DC component from each of the plurality of input image data to create a plurality of DC images, A procedure for detecting the relative positions of the plurality of DC images and determining whether the detected relative positions are within a predetermined range suitable for panorama synthesis, and determining whether the detected relative positions are within the predetermined range from the plurality of image data. A step of selecting only a plurality of image data, a step of decoding the selected plurality of image data, and a step of combining the decoded plurality of image data based on the detected relative position to create a panoramic image Procedures are provided.

Further, in the storage medium according to the present invention, a process of inputting a plurality of image data, a process of extracting a DC component from each of the plurality of input image data to create a plurality of DC images, DC
A process of detecting a relative position of an image; determining whether the detected relative position is within a predetermined range suitable for panorama synthesis; and determining a plurality of image data within the predetermined range from the plurality of image data. Executing a process of selecting only a plurality of image data, a process of decoding the selected plurality of image data, and a process of combining the decoded plurality of image data based on the detected relative position to create a panoramic image The program for performing is stored.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. (First Embodiment) FIG. 1 is a block diagram showing a panoramic image creating apparatus according to a first embodiment of the present invention. In the present embodiment, it is possible to use up to four images for panoramic synthesis. For convenience, input image data will be referred to as first compressed image data, second compressed image data, third compressed image data, and fourth compressed image data in order. In each compressed image data, pixels on one screen are divided into N × N small blocks, and each block is subjected to orthogonal transformation.

In FIG. 1, reference numeral 10 denotes an input unit for inputting an image, 20 denotes a changeover switch, 30, 40, 50, and 60.
Is a memory for holding the input compressed image data (hereinafter, referred to as memories 1, 2, 3, and 4), 70 is a changeover switch, 80 is a DC image creation unit that creates a DC image from the compressed image data, and 90 is Changeover switch, 100, 1
Reference numeral 10 denotes a DC image memory for holding a DC image, 120
Is a relative position detection unit that detects a relative position, 130 is an image selection unit that selects an image, 140 is a relative position data unit, 15
Reference numeral 0 denotes a decoding unit, 160 denotes an image synthesizing unit, and 170 denotes a panoramic memory for holding the synthesized image.

Next, the operation of this system will be described. First, the changeover switch 20 is connected to A, and the changeover switch 70 is connected to E. First, a process of inputting all the compressed image data and storing it in a memory in the apparatus is performed. The input unit 10 performs a process of acquiring the first compressed image data, and passes the acquired compressed image data to the changeover switch 20. Further, after transmitting the compressed image data to the changeover switch 20, the input unit 10 performs control to switch the changeover switch 20 next so that all the input compressed image data can be held. That is, in this case, control is performed so that the switch is connected to B.

The changeover switch 20 is connected to A, and when receiving compressed image data from the input unit 10, stores the compressed image data in the memory 1. Thereafter, the changeover switch 20 is connected to B by the input unit 10. Next, when the second compressed image is input, the input unit 10 similarly obtains the compressed image data, and passes the compressed image data to the changeover switch 20. After transmitting the data to the changeover switch 20, the changeover switch 20 is set to C
Control to connect to.

When the changeover switch 20 receives the second compressed image data from the input unit 10, it stores it in the memory 2. Thereafter, the changeover switch 20 is connected to C by the input unit 10. Similarly, for the input of the third and fourth compressed images, the input unit 10 and the changeover switch 20
Performs a process of storing the input image in the memories 3 and 4 while switching between C and D.

After the process of reading all the compressed images into the memory is completed, a process of comparing the respective images is performed. First, in order to compare the first compressed image data with the second compressed image data, a process of creating a DC image of the first compressed image data is performed. The DC image creation unit 80 acquires the first compressed image data stored in the memory 1 through the changeover switch 70. In each compressed image data, pixels on one screen are divided into N × N small blocks, and each block is subjected to orthogonal transformation. DC of each block
An image obtained by extracting only components is called a DC image.

Upon acquiring the compressed image data, the DC image creating section 80 performs a process of extracting the DC components of the orthogonally transformed blocks and creating a DC image. Then, the DC image creation unit 80 writes the DC image of the first image data into the DC image memory 100 through the changeover switch 90. Thereafter, the changeover switch 70 is connected to F and the changeover switch 90 is connected to J. Next, the DC image creation unit 80
Performs a process of obtaining the second compressed image data from the memory 2 through the changeover switch 70, extracting the DC component of each orthogonally transformed block, and creating a DC image.
Then, the DC image creation unit 80 sets the changeover switch 90
Then, the DC image of the second compressed image data is written to the DC image memory 110 through the. Then, the changeover switch 70
To G.

When the DC image of the two compressed image data can be obtained, the relative position detecting unit 120
DC images are acquired from 0 and 110, and the relative positions of the two images are detected from those data. The relative position detection process can be detected by using a known general method such as pattern matching or using a velocity vector.

When the relative positions of the two images are detected, the relative position detecting section 120 converts the relative position data into the image selecting section 13.
Send to 0. Upon receiving the relative position data, the image selecting unit 130 performs a process of determining whether or not the image is an image necessary for the panorama synthesizing process from the data. For example, if the overlap area of two images is 30% or more of the image pixels, it is determined that the image is unnecessary for composition, and if less than 30%, it is determined that it is necessary.

When the image selection unit 130 determines that an image is necessary, the relative position data is stored in the relative position data unit 140. When the image selecting unit 130 determines that an image is necessary, the image selecting unit 130 switches the switch 90 to connect to I. If it is determined that it is unnecessary, the switch 90 is connected to J without switching.

Next, the image selection unit 130 transmits a DC image creation command for the next image to the DC image creation unit 80. next,
When it is determined that the second compressed image data is unnecessary for the panorama synthesizing process, a comparison process between the first compressed image data and the third compressed image data is performed. If it is determined that the second compressed image data is necessary for the panorama synthesizing process,
A comparison process is performed between the second compressed image data and the third compressed image data.

The DC image creating section 80 acquires the third compressed image data from the memory 3 through the changeover switch 70 and performs a process of creating a DC image from the compressed image data. At this time, the changeover switch 90 is connected to the image selection unit 13.
When 0 determines that the second image is necessary, it is connected to I. When it is determined that the second image is unnecessary, it is connected to J.

When the changeover switch 90 is connected to I, the DC image creating section 80 sets the changeover switch 90
And writes the DC image of the third compressed image data into the DC image memory 100. After that, the changeover switch 7
Connect 0 to H. When the changeover switch 90 is connected to J, the DC image creation unit 80 writes the DC image of the third compressed image data into the DC image memory 110 through the changeover switch 90. Thereafter, the changeover switch 70 is connected to H.

That is, when the changeover switch 90 is connected to I, the DC image of the third compressed image data is stored in the DC image memory 100, and the second image is stored in the DC image memory 110.
Of the compressed image data is stored. When the changeover switch 90 is connected to J, the DC image of the first compressed image data is stored in the DC image memory 100, and the DC image of the third compressed image data is stored in the DC image memory 110. Will be.

Each time the changeover switch 70 is switched from E to H, the first to fourth compressed image data from each of the memories 1 to 4 are sent to the decoding unit 150 and held.

Next, the relative position detection unit 120 acquires each DC image from the DC image memories 100 and 110,
The relative positions of the two images are detected from the data. When the relative positions of the two images are detected, the relative position detector 120 transmits the relative position data to the image selector 130.

Upon receiving the relative position data, the image selecting section 130 performs a process of judging from the data whether or not the image is necessary for the panorama synthesizing process. For example, if the overlap area of two images is 30% or more of the image pixels, it is determined that the image is unnecessary for synthesis, and if less than 30%, it is determined that it is necessary.

When the image selecting section 130 determines that an image is necessary, it stores the relative position data in the relative position data section 140 and switches the changeover switch 90.
When it is determined that the image is unnecessary, the changeover switch 90 is left as it is without switching.

Next, the image selecting unit 130 transmits a DC image creating command for the next image to the DC image creating unit 80. As in the previous case, the fourth compressed image data
A C image creation process is performed, and a relative position detection process is performed. When the relative position detection processing and the image selection processing have been completed for all the image data, the relative position data unit 140 holds the relative position data of all the images determined to be necessary for the panorama synthesis processing.

When the relative position detection processing and the image selection processing are completed for all the image data, the panorama synthesis processing is performed next. The image synthesizing unit 160 acquires relative position data of image data necessary for panorama synthesis from the relative position data unit 140. Next, the image synthesizing unit 160 requests the decoded image data necessary for the decoding unit 150 to obtain decoded image data of all images in the relative position data.

The decoding unit 150 acquires compressed image data of a plurality of images requested by the image synthesizing unit 160 from the stored compressed image data, performs a decoding process, and decodes the decoded image data into the image synthesizing unit 160. Pass to. Upon receiving all the decoded image data required for the panorama synthesis process, the image synthesis unit 160 performs an image synthesis process using the relative position data of each image held in the relative position data unit 140. At that time, a color tone smoothing process generally used in the image synthesizing process may be performed. The created combined data is stored in the panorama memory 170. With the above configuration and operation, a quick panorama composition process can be performed.

Next, the operation of the system of this embodiment will be described with reference to the flowchart of FIG. In FIG. 2, a step S10 performs a process of inputting a compressed image. A step S20 performs a process of storing the input compressed image in the memory. A step S30 decides whether or not all the images have been input. If there are still images to be input, the process proceeds to step S10, and if all images have been input, the process proceeds to S40. Steps S10, S20, and S30 correspond to the input unit 10 in FIG.

When all the images have been input, the process proceeds to step S40. A step S40 performs a process of creating a DC image of the first compressed image. Step S50 is
A process of creating a DC image of the next compressed image is performed. Steps S40 and S50 correspond to the DC image creation unit 80 in FIG.

In step S60, a relative position detection process is performed by comparing the DC images of the two compressed image data. This step S60 corresponds to the relative position detector 120 in FIG. A step S70 decides whether or not the current target image is necessary for the panorama synthesizing process based on the relative position detected in the step S60. If it is determined in step S70 that an image is necessary, the process proceeds to step S90. If it is determined in step S70 that the image is unnecessary, the process proceeds to step S100.

In step S90, a process for storing the relative position data of the image determined to be necessary in step S70 is performed. In step S100, the process proceeds to step S110 if all images have been compared, and to step S50 if there are still images to be compared. Steps S70, S80, S90, S100
Corresponds to the image selection unit 130 in FIG.

In step S110, an image selected as necessary for the panorama synthesizing process is decoded. This step S110 corresponds to the decoding section 150 in FIG. In step S120, the image decoded in step S110 is obtained, and a panorama image combining process is performed based on the relative position information. This step S120 corresponds to the image synthesizing section 160 in FIG.

In this embodiment, the panorama image synthesis of a maximum of four images is performed using the memories 1 to 4. However, the panorama image synthesis of an arbitrary number of images is possible by increasing the number of memories. Needless to say,
It goes without saying that the input image may be composed of a plurality of continuous image data such as moving image data.

According to the present embodiment, images are compared based on their DC images without decoding a plurality of compressed image data, so that an image required for panorama synthesis can be selected quickly. In addition, it is possible to quickly perform the alignment at the time of the panorama synthesis, and to shorten the panorama synthesis processing time.

(Second Embodiment) FIG. 3 shows a second embodiment of the present invention.
An embodiment will be described. In FIG. 3, the selection condition flag and the high-precision relative position detection unit 180 are added to FIG. 1, and the other parts corresponding to those in FIG. The description of the operation will be omitted. In the present embodiment, the selection condition flag is input to the image selection unit 130 so that the conditions for image selection can be changed.

The image selecting section 130 first obtains a selection condition flag for image selection, and determines an image selection condition according to the value. For example, if the selection condition flag is 0, the image selection unit 130 performs processing so as to determine that the overlap area of the two images is necessary if the overlap area is less than 10% of the image pixels, and that the other areas are unnecessary. . Further, if the selection condition flag is 1, processing is performed so as to determine that it is necessary if the value is less than 20%, and unnecessary if it is not the other. If the selection condition flag is 2,
If it is less than 30%, processing is performed so that it is determined that it is necessary, and otherwise it is unnecessary.

As described above, the configuration for freely selecting the image selection condition can be changed by the selection condition flag. In this example, the condition of the image selection can be changed by the selection condition flag. However, it is also possible to specifically change the condition of the image selection by inputting the percentage number or the number of pixels of the overlap area. good.

After first determining the conditions for image selection, the image selection section 130 performs the same processing as in the first embodiment. Next, the relative position data unit 140 holds the relative position data of each image. When the relative position detection processing and the image selection processing have been completed for all the image data, the relative position data unit 140 holds the relative position data of all the images determined to be necessary for the panorama synthesis processing.

When the relative position detection processing and the image selection processing have been completed for all the image data, high precision relative position detection processing is performed next. The high-precision relative position detection unit 180 acquires relative position data of image data necessary for panorama synthesis from the relative position data unit 140. High precision relative position detector 18
0 requests the decoding unit 150 for necessary decoded image data in order to obtain decoded image data of all images in the relative position data. The decoding unit 150 acquires the compressed image data of the requested image, performs a decoding process, and passes the decoded image data to the high-accuracy relative position detection unit 180 and the image synthesis unit 160.

Upon acquiring the relative position data and the decoded image data, the high-precision relative position detection unit 180 performs a process of detecting the relative position with respect to each image with high accuracy. When the high-precision relative position detection unit 180 completes the detection of the high-precision relative position, the high-precision relative position detection unit 180 passes the high-precision relative position data to the image synthesis unit 160. The image synthesizing unit 160 includes the decoding unit 15
In addition to obtaining decoded image data of an image necessary for panorama synthesis from 0, obtaining high-precision relative position data from the high-precision relative position detection unit 180, and performing image synthesis processing based on the high-precision relative position data. Do. At this time, a color tone smoothing process generally used in the image synthesizing process is performed. The created combined data is stored in the panorama memory 17.
0 is stored.

Next, the operation of the system of this embodiment will be described with reference to the flowchart of FIG. The flowchart in FIG. 4 is obtained by adding step S301 after step S30 in FIG. 2 and adding step S1101 after step 110.

In step S301, a process for inputting a selection condition flag and determining a condition for image determination based on the selection condition flag is performed. In step S1101, the image decoded in step S110 is acquired, and a process of detecting a highly accurate relative position of each image based on the relative position information is performed. Next, in step S120, a panoramic image synthesizing process is performed on the decoded image based on the detected high-accuracy relative position information.

According to the present embodiment, it is possible to compare images without decoding the images, to enable quick panorama synthesis processing, and to select positioning accuracy in panorama synthesis. As a result, it is possible to perform image synthesis with high accuracy according to the image content and the like, and it is possible to obtain a high-quality panoramic image in a short processing time.

Next, a storage medium according to another embodiment of the present invention will be described. The system shown in FIGS. 1 and 3
Although it can be constituted by hardware, it can be constituted by a computer system including a CPU and a memory. When constituted by a computer system, the memory constitutes a storage medium according to the present invention. The storage medium stores a program for executing the processing procedure for controlling the operation described in each of the flowcharts of FIGS. 2 and 4 according to each of the above embodiments.

As the storage medium, ROM, R
Semiconductor memory such as AM, optical disk, magneto-optical disk,
A magnetic medium or the like may be used, and these may be configured and used in a CD-ROM, a floppy disk, a magnetic tape, a magnetic card, a nonvolatile memory card, or the like.

Therefore, this storage medium is used in a system or apparatus other than those shown in FIGS. 1 and 3, and the system or computer reads out and executes the program code stored in this storage medium. Accordingly, the same functions as those of the above embodiments can be realized, the same effects can be obtained, and the object of the present invention can be achieved.

An OS running on a computer
Perform part or all of the processing, or after the program code read from the storage medium is written into the memory provided in the extended function board inserted into the computer or the extended function unit connected to the computer. ,
Even when the CPU or the like provided in the above-mentioned extended function board or extended function unit performs a part or all of the processing based on the instruction of the program code, the same functions as those of the above embodiments can be realized and the same functions can be realized. The effect can be obtained, and the object of the present invention can be achieved.

[0053]

As described above, according to the present invention,
By comparing each image using a DC image created from the plurality of image data without decoding the image data, it is possible to quickly select an image required for panorama synthesis and to perform panorama synthesis. Can be quickly performed, so that the panorama synthesis processing time can be reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a panoramic image creating device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing an operation of the first embodiment.

FIG. 3 is a block diagram showing a panoramic image creating device according to a second embodiment of the present invention.

FIG. 4 is a flowchart showing the operation of the second embodiment.

FIG. 5 is a configuration diagram illustrating an outline of a panoramic image synthesizing process.

[Explanation of symbols]

 Reference Signs List 10 input unit 20 changeover switch 30, 40, 50, 60 memory 1-47 changeover switch 80 DC image creation unit 90 changeover switch 100, 110 DC image memory 120 relative position detection unit 130 image selection unit 140 relative position data unit 150 decoding Unit 160 Image synthesis unit 170 Panorama memory 180 High-precision relative position detection unit

Claims (8)

[Claims] An input unit for inputting a plurality of image data; and a DC for each of the plurality of input image data.
DC image creating means for extracting a component to create a plurality of DC images; comparing means for comparing the created plurality of DC images; and panoramic synthesis from the plurality of image data based on a comparison result of the comparing means. Selecting means for selecting a plurality of necessary image data; decoding means for decoding the selected plurality of image data; and combining the decoded plurality of image data based on a comparison result of the detected comparing means. And a synthesizing means for creating a panoramic image. 2. The method according to claim 1, wherein the comparing unit includes a relative position detecting unit that detects a relative position of the plurality of DC images, and the selecting unit determines that the detected relative position is a predetermined position suitable for creating the panoramic image. 2. The panoramic image creating apparatus according to claim 1, wherein it is determined whether the image data is within the range, and only a plurality of image data within the predetermined range are selected. 3. A plurality of predetermined ranges suitable for creating the panoramic image can be set, and a designating means for designating one of the plurality of predetermined ranges is provided. 3. A panoramic image creating apparatus according to claim 1, wherein a plurality of image data corresponding to the image data are selected. 4. A high-accuracy relative-position detecting means for detecting a more accurate relative position from the decoded image data based on the detected relative position, wherein the synthesizing means comprises: 4. The panoramic image creating apparatus according to claim 2, wherein the image combining is performed based on the relative position. 5. The method according to claim 1, wherein the plurality of input image data are image data compressed using orthogonal transform in block units, and the DC image is created from DC components of each block. The panoramic image creating device according to claim 1. 6. The panoramic image creating apparatus according to claim 1, wherein said plurality of image data is moving image data. 7. A procedure for inputting a plurality of image data, and DC
Extracting a component to create a plurality of DC images, detecting relative positions of the plurality of DC images, and determining whether the detected relative positions are within a predetermined range suitable for panorama synthesis A step of selecting only a plurality of image data within the predetermined range from the plurality of image data; a step of decoding the selected plurality of image data; and a step of detecting the decoded plurality of image data. Creating a panoramic image by synthesizing based on the obtained relative position. 8. A process for inputting a plurality of image data, and a DC process for each of the plurality of input image data.
A process of extracting a component to create a plurality of DC images, a process of detecting a relative position of the plurality of DC images, and determining whether the detected relative position is within a predetermined range suitable for panorama synthesis A process of selecting only a plurality of image data within the predetermined range from the plurality of image data; a process of decoding the selected plurality of image data; and a process of detecting the decoded plurality of image data And a computer-readable storage medium storing a program for executing a process of creating a panoramic image by combining based on the obtained relative position.