JPH11328443A - System and method for generating three-dimensional panorama image and recording media therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system for generating a panorama image with which three-dimensional graphic data with texture are synthesized. SOLUTION: An image inputting part 1 inputs actually photographed background image data. A processing controlling part 4 connects the inputted background image data for generating one panorama image. The image inputting part 1 inputs object image data constituted of three-dimensional graphic data. The processing control part 4 inputs light source information being light source setting substantially equal to that of the background image data through an operation inputting part 2. The processing control part 4 renders the object image data for applying texture to the data according to the inputted light source information. The processing controlling part 4 synthesizes the rendered object image data with the panorama image, and an image displaying part 6 displays the synthesized panorama image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional panoramic image generating system for generating a virtual three-dimensional panoramic image by synthesizing three-dimensional graphic data with an image actually picked up by an image pickup device or the like. The present invention relates to a panoramic image generation method and a recording medium.

[0002]

2. Description of the Related Art Heretofore, there has been known a technique of generating a virtual image in which graphic data of a three-dimensional building or the like generated by computer graphics or the like is synthesized with a landscape image actually captured by an imaging device such as a camera. Have been. This virtual image is used in landscape verification, architectural presentation, and the like, and can easily clarify a landscape or the like after completion of a building or the like.

On the other hand, recently, a landscape image or the like is displayed on a display device such as a personal computer, and a display image is changed in accordance with a user's instruction to change a viewpoint position to display a 360-degree three-dimensional panoramic image. There are known three-dimensional panoramic image technologies that can be used. This three-dimensional panoramic image technique is somewhat different from conventional panoramic images. Whereas conventional panoramic images display all panoramic images, 3D panoramic image technology displays a partial area of the panoramic image and smoothly moves the image according to a user's viewpoint change instruction. , For displaying a three-dimensional panoramic image of 360 degrees.

Hereinafter, such a three-dimensional panoramic image technique will be described with reference to the drawings. As shown in FIG. 13, a case will be described in which a plurality of images captured by changing the angle by an imaging device such as a camera and having a center point determined are panoramic. FIG. 14 shows a plurality of images captured in this manner. According to the three-dimensional panoramic image technology, such a plurality of images are pasted inside an annular band in the virtual space shown in FIG. 15 and are joined so that joints are inconspicuous, thereby forming one three-dimensional panoramic image. Generate. The three-dimensional panoramic image technology displays a partial area in the three-dimensional panoramic image generated in this way, and smoothly moves the image according to a user's viewpoint change instruction to form a three-dimensional panoramic image of 360 degrees. To display. At present, there is a movement to utilize such a useful three-dimensional panoramic image for landscape verification and architectural presentation.

[0005]

However, there are some problems in using a three-dimensional panoramic image for landscape verification and architectural presentation. In practice, the three-dimensional panoramic image is obtained by unwinding an annular band in the virtual space, converting the position to a two-dimensional plane, and storing the converted position in a memory such as a personal computer. The image in the two-dimensional area in the memory is distorted. That is, the image of the table in the three-dimensional panoramic image as shown in FIG. 16A is distorted as shown in FIG. 16B. Therefore, when synthesizing graphic data in landscape verification or the like, it is necessary to generate distorted graphic data and synthesize it with a panoramic image. Further, since the distortion of the image in the panoramic image differs depending on the place, it is necessary to adjust the distortion depending on where the graphic data is arranged and combined in the panoramic image. Furthermore, in order to faithfully reproduce the illumination and shade of light on the same level as the panorama image serving as the background image by actual shooting in graphic data, and to express it accurately and realistically,
Some experience was required.

SUMMARY OF THE INVENTION The present invention has been made in view of the above situation, and has a three-dimensional panorama image generation system and a three-dimensional panorama image generation system capable of generating a panorama image in which textured three-dimensional graphic data is synthesized. It is an object to provide a method and a recording medium.

[0007]

In order to achieve the above object, a three-dimensional panoramic image generating system according to a first aspect of the present invention comprises a background image input system for inputting a plurality of image data as a real image of a background image. Means, a plurality of image data input by the background image input means, a panoramic image generating means for generating one panoramic image by combining the plurality of image data, and an object data comprising three-dimensional graphic data. Object data input means, and a light source for inputting, to the object data input by the object data input means, light source information having at least substantially the same light source settings as the image data input by the background image input means Information input means, and the panoramic image generating means according to the light source information input by the light source information input means. Characterized in that it comprises synthesizing means for synthesizing the object data input in the object data input means to the panoramic image generated, and display means for displaying a panoramic image synthesized by said synthesizing means by.

According to the present invention, the panoramic image generating means combines a plurality of image data input by the background image input means to generate one panoramic image. The light source information input means inputs, to the object data input by the object data input means, light source information having light source settings substantially the same as at least the image data input by the background image input means. The combining means combines the object data input by the object data input means with the panoramic image generated by the panoramic image generating means according to the light source information input by the light source information input means. The display means displays the panoramic image synthesized by the synthesis means. As a result, it is possible to generate graphic data of a textured three-dimensional shape that is substantially the same as the panorama image that is the actually captured background image.

A coordinate position setting means for setting a coordinate position in a virtual space for synthesizing the object information input by the object data input means with the image information of the panorama image generated by the panorama image generation means; Distortion information acquiring means for acquiring at least distortion information in the panoramic image at the coordinate position set by the position setting means, wherein the synthesizing means generates the panoramic image according to the distortion information acquired by the distortion information acquiring means. The object data input by the object data input means may be combined with the panoramic image generated by the means. As a result, it is possible to generate graphic data of a textured three-dimensional shape that is substantially the same as the panorama image that is the actually captured background image.

[0010] The light source information input means inputs photographing information comprising at least longitude information, latitude information, time information and weather information. According to the inputted photographing information, the object data input by the object data input means is input to the object data. Light source information having substantially the same light source setting as the image data input by the background image input means may be input. As a result, it is possible to generate graphic data of a textured three-dimensional shape that is substantially the same as the panorama image that is the actually captured background image.

[0011] The image processing apparatus may further include pixel complementing means for complementing pixels at the boundary of the object data combined in the panoramic image combined by the combining means.

In order to achieve the above object, a second aspect of the present invention is provided.
The method for generating a three-dimensional panoramic image according to the aspect of the present invention comprises: combining a plurality of image data input in the background image input step; and a plurality of image data input in the background image input step. A panoramic image generating step of generating a single panoramic image, and an object data inputting step of inputting object data composed of three-dimensional graphic data;
A light source information input step of inputting light source information having substantially the same light source setting as the image data input in the background image input step, to the object data input in the object data input step,
A combining step of combining the object data input in the object data input step with the panorama image generated in the panorama image generating step, according to the light source information input in the light source information input step; A display step of displaying a panoramic image.

According to the present invention, in the panoramic image generation step, a plurality of image data input in the background image input step are respectively combined to generate one panoramic image. In the light source information input step, light source information having substantially the same light source setting as at least the image data input in the background image input step is input to the object data input in the object data input step.
The synthesizing step synthesizes the object data input in the object data input step with the panoramic image generated in the panoramic image generating step according to the light source information input in the light source information input step.
The display step displays the panoramic image synthesized in the synthesis step. As a result, it is possible to generate graphic data of a textured three-dimensional shape that is substantially the same as the panorama image that is the actually captured background image.

In order to achieve the above object, a third aspect of the present invention is provided.
The recording medium according to the aspect of the present invention is a computer-readable recording medium that records a program that causes a computer to function as a three-dimensional panoramic image generation system, and inputs a plurality of image data to be a captured real background image. A panoramic image generating step of combining each of the plurality of image data input in the background image input step to generate one panoramic image;
An object data input step of inputting object data composed of graphic data of a three-dimensional shape; and an object data input in the object data input step, wherein at least the image data input in the background image input step A light source information input step of inputting light source information having the same light source setting, and a panorama image generated by the panorama image generation step according to the light source information input in the light source information input step, in the object data input step. A program for realizing a synthesizing step of synthesizing the input object data and a display step of displaying a panoramic image synthesized by the synthesizing step is stored.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a three-dimensional panoramic image generating system according to an embodiment of the present invention will be described with reference to the drawings. The three-dimensional panoramic image generation system shown in FIG. 1 includes an image input unit 1, an operation input unit 2, a parameter storage unit 3, a processing control unit 4, an image storage unit 5, and an image display unit 6. You.

An image input unit 1 inputs a plurality of background image data obtained by actual photographing, and an image storage unit 5 via a processing control unit 4.
To memorize. Further, the image input unit 1 inputs an object image composed of three-dimensionally shaped graphic image data, and stores the object image in the image storage unit 5.

The operation input unit 2 includes an input device such as a keyboard and a mouse, etc., acquires instruction data for generating a three-dimensional panoramic image input by a user, and the like.
To supply.

The parameter storage section 3 stores various parameters used in rendering processing for giving a texture to the object image performed by the processing control section 4.

The processing control unit 4 performs processing, light source setting, and rendering for the object image. Further, the processing control unit 4 combines the background image data and the object image data. In addition, the processing control unit 4
A combining process for combining the object image data with the panoramic image data is performed.

The image storage unit 5 comprises a memory or the like, and stores background image data and object image data input from the image input unit 1. Further, the image storage unit 5 stores the panoramic image data after the synthesis processing generated by the processing control unit 4.

The image display unit 6 is a CRT (Cathode Ray Tu)
be) and the like, and displays the panoramic image data stored in the image storage unit 5.

Hereinafter, a three-dimensional panorama image generation process in the three-dimensional panorama image generation system according to the embodiment of the present invention will be described with reference to the drawings. FIG. 2 is a flowchart illustrating a three-dimensional panorama image generation process performed by the three-dimensional panorama image generation system. When the user inputs instruction data for instructing start of the panorama image creation generation process from the operation input unit 2, the three-dimensional panorama image generation process illustrated in FIG. 2 starts.

First, the image input unit 1 inputs a plurality of background image data from an external color scanner or the like (not shown) (step S1). That is, the image input unit 1 inputs a plurality of background image data as shown in FIG. The image input unit 1 supplies the input background image data to the processing control unit 4. The processing control unit 4 stores the acquired background image data in the image storage unit 5.

The processing control section 4 combines the background image data stored in the image storage section 5 with a panoramic image (step S).
2). That is, the processing control unit 4 pastes a plurality of background image data as shown in FIG. 3 inside the annular band in the virtual space and combines them so that the joints are not conspicuous to form one panoramic image. Combine. The processing control unit 4 stores the panoramic image converted into a two-dimensional plane as shown in FIG. 4 in the image storage unit 5 by unwinding the annular band in the virtual space.

Next, the image input unit 1 inputs three-dimensional object data composed of polygons and the like from an external computer graphics generation device or the like (not shown) (step S3). That is, the image input unit 1 is configured as shown in FIG.
Input object data which is graphic data of a three-dimensional building as shown in FIG. The image input unit 1
The input object data is supplied to the processing control unit 4. The processing control unit 4 stores the acquired object image data in the image storage unit 5.

When the object image data is stored in the image storage unit 5, the processing control unit 4 arranges the object image data on the plane coordinates in the virtual space according to the instruction data from the user (step S4). Hereinafter, the process of arranging the object data on the plane coordinates performed by the process control unit 4 will be described in detail with reference to the flowchart shown in FIG.

The processing control unit 4 displays a plane coordinate map such as a topographic map of a plane as shown in FIG. 7 previously stored in the image storage unit 5 corresponding to the background image data shown in FIG. (Step S41).

The operation input unit 2 inputs instruction data from a user and supplies the data to the processing control unit 4. The processing control unit 4
According to the instruction data obtained from the operation input unit 2, the object data is arranged on the plane coordinate diagram (step S4).
2). That is, the processing control unit 4 arranges the object data, for example, at a place where a building is to be constructed according to the instruction data.

The processing control section 4 generates an object display image viewed from a predetermined viewpoint position according to the coordinate position of the arranged object data on the plane coordinate diagram (step S43).

The processing control section 4 displays the generated object display image on the image display section 6 (step S44).

Returning to FIG. 2, when the position of the object on the plane coordinates in the virtual space is determined in step S4, the processing control unit 4 processes the object data in accordance with the instruction data from the user (step S5). .
Hereinafter, the processing of object data performed by the processing control unit 4 will be described in detail with reference to the flowchart shown in FIG.

The operation input unit 2 inputs instruction data from a user and supplies the instruction data to the processing control unit 4. The processing control unit 4
The size and display direction of the object image data are changed according to the instruction data obtained from the operation input unit 2 (step S51).

Further, the processing control section 4 changes the color, pattern, etc. set in the object data according to the instruction data obtained from the operation input section 2 (step S5).
2). That is, the processing control unit 4 changes the color information, the specular reflection coefficient information, the transmission coefficient information, and the texture information of the object data according to the acquired instruction data.

The processing control section 4 generates an object display image obtained by viewing the changed object data from a predetermined viewpoint position (step S53). Processing control unit 4
Displays the generated object image on the image display unit 6 (step S54).

Returning to FIG. 2, when the processing of the object data is performed in step S5, the processing control unit 4
Sets the light source according to the instruction data from the user (step S6). Hereinafter, the light source setting processing of the object data performed by the processing control unit 4 will be described in detail with reference to the flowchart shown in FIG.

When the background data stored in the image storage unit 5 is an outdoor image, the processing control unit 4 displays a light source information input screen as shown in FIG. 10 on the image display unit 6 (step S61). ).

The operation input unit 2 inputs light source information from a user and supplies the light source information to the processing control unit 4. The processing control unit 4 generates the light source setting data according to the light source information acquired from the operation input unit 2 (Step S62). That is, the processing control unit 4 calculates the position of the sun serving as a light source, the intensity of light from the sun, and the like according to the parameter information stored in the parameter storage unit 3 according to the acquired light source information, and sets the light source setting data. Generate.

The processing control section 4 stores the generated light source setting data in the image storage section 5 (step S63).

Returning to FIG. 2, in step S6, when the light source setting processing of the object data is performed, the processing control unit 4 renders the object data in accordance with the generated light source setting data so as to give the object data a texture close to a real photograph. Is performed (step S7). Hereinafter, the rendering processing and the combining processing of the object data performed by the processing control unit 4 will be described in detail with reference to the flowchart shown in FIG.

The processing control unit 4 determines the light source according to the light source setting data stored in the image storage unit 5 in step S63 in FIG.
The color and pattern set in the object data are changed (step S71). That is, the processing control unit 4
The color, specular reflection coefficient, transmission coefficient, and texture data of the object image are changed based on the setting data of the light source generated by ray tracing (ray tracing method) or the like.

The processing control section 4 deforms the object data having the texture close to the actual image by giving the same distortion as the background image data corresponding to the position on the plane coordinates arranged in step S4 ( Step S72). That is, the processing control unit 4 acquires the distortion of the background image data corresponding to the position on the plane coordinates where the object data is arranged, and deforms the object data according to the acquired distortion.

The processing control section 4 combines the object data and the panoramic image data (step S73). That is, the processing control unit 4 combines the object data with the panoramic image data generated in step S2. The processing control unit 4 performs anti-aliasing to make the edge part, which is the boundary of the object data in the synthesized panoramic image data, inconspicuous (step S74). That is, the processing control unit 4 makes the edge of the object data a complementary color with an intermediate color between adjacent colors to make the boundary inconspicuous. As a result, the processing control unit 4 generates a panoramic image in which a building serving as object image data as shown in FIG. 12 is synthesized.

The processing control section 4 displays the generated panoramic image on the image display section 6 (step S8). That is,
The processing control unit 4 displays a partial area of the generated panoramic image, and smoothly moves the image in accordance with the user's viewpoint change instruction to display a 360-degree three-dimensional panoramic image. As a result, light from a light source that is substantially the same as the panoramic image is given to the object data, and a three-dimensional panoramic image with a texture similar to a real image can be generated.

In the above-described embodiment, a plurality of background image data input from the image input unit 1 are subjected to panoramic synthesis by the processing control unit 4. However, panorama synthesis may be input from the beginning. The processing can be speeded up by using the panorama synthesis generated by another image generation device.

It should be noted that the three-dimensional panoramic image generation system of the present invention can be realized using an ordinary computer system without using a dedicated system. For example, a medium (floppy disk, CD-RO) storing a program for executing any of the above-described processes in a computer
M, etc.), a three-dimensional panoramic image generation system that executes the above-described processing can be configured.

The medium for supplying the program to the computer may be a communication medium (medium for temporarily and fluidly storing the program, such as a communication line, a communication network, or a communication system). For example, the program may be posted on a bulletin board (BBS) of a communication network and distributed via the network. Then, by starting this program and executing it in the same manner as other application programs under the control of the OS, the above-described processing can be executed.

[0047]

As described above, according to the present invention,
It is possible to generate a panoramic image in which graphic data of a textured three-dimensional shape is synthesized.

[Brief description of the drawings]

FIG. 1 is a schematic diagram illustrating a configuration of a three-dimensional panoramic image generation system according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a three-dimensional panoramic image generation process according to the embodiment of the present invention.

FIG. 3 is a schematic diagram showing a plurality of background images.

FIG. 4 is a schematic diagram showing a panoramic image synthesized from a background image.

FIG. 5 is a schematic diagram illustrating an example of object image data.

FIG. 6 is a flowchart illustrating an arrangement process of object image data.

FIG. 7 is a schematic diagram showing an example of a plane coordinate map including a plane topographic map and the like.

FIG. 8 is a flowchart illustrating processing for processing object image data.

FIG. 9 is a flowchart illustrating a light source setting process.

FIG. 10 is a schematic diagram illustrating an example of a light source information input screen.

FIG. 11 is a flowchart illustrating a rendering process and a combining process.

FIG. 12 is a schematic diagram showing a completed three-dimensional panoramic image.

FIG. 13 is a schematic diagram showing a state in which shooting is performed while changing the angle.

FIG. 14 is a schematic diagram showing a plurality of captured images.

FIG. 15 is a schematic diagram showing an image of an image pasted inside a ring-shaped band in a virtual space.

FIG. 16 is a schematic diagram showing a normal image and a distorted image.

[Explanation of symbols]

 1 image input unit 2 operation input unit 3 parameter storage unit 4 processing control unit 5 image storage unit 6 image display unit

Claims (6)

[Claims] 1. A background image input means for inputting a plurality of image data serving as a photographed background image, and a plurality of image data input by the background image input means are respectively combined to form one panoramic image. Panoramic image generating means for generating, object data input means for inputting object data composed of three-dimensional graphic data, object data input by the object data input means, at least the background image input means Light source information inputting means for inputting light source information having substantially the same light source setting as the image data input by the user, and a panoramic image generated by the panoramic image generating means according to the light source information input by the light source information inputting means The object data input by the object data input means. A three-dimensional panoramic image generation system, comprising: a synthesizing unit configured to generate a panoramic image synthesized by the synthesizing unit. 2. A coordinate position setting means for setting a coordinate position in a virtual space for synthesizing object data input by the object data input means with image information of a panorama image generated by the panorama image generation means; Distortion information acquiring means for acquiring at least distortion information in a panoramic image at the coordinate position set by the coordinate position setting means, wherein the synthesizing means is configured to perform the panorama according to the distortion information acquired by the distortion information acquiring means. The three-dimensional panoramic image generation system according to claim 1, wherein the panorama image generated by the image generation unit is combined with the object data input by the object data input unit. 3. The light source information input means inputs photographing information comprising at least longitude information, latitude information, time information and weather information, and outputs the object data inputted by the object data input means in accordance with the inputted photographing information. 3. The three-dimensional panoramic image generation system according to claim 1, wherein light source information having substantially the same light source setting as the image data input by the background image input means is input to the three-dimensional panoramic image generation system. 4. The three-dimensional image according to claim 1, further comprising a pixel complementing unit that complements a boundary of the object data combined in the panoramic image combined by the combining unit with a pixel. Panoramic image generation system. 5. A background image inputting step of inputting a plurality of image data serving as a background image actually captured, and combining a plurality of image data input in the background image inputting step to form one panoramic image. A panoramic image generating step of generating; an object data inputting step of inputting object data composed of three-dimensional graphic data; and an object data input in the object data inputting step, at least in the background image inputting step. Light source information inputting step of inputting light source information having substantially the same light source setting as the image data input in the step (a), and a panoramic image generated in the panoramic image generating step in accordance with the light source information input in the light source information inputting step The object data input step A three-dimensional panoramic image generation method, comprising: a synthesizing step of synthesizing object data input in a step; and a display step of displaying a panoramic image synthesized by the synthesizing step. 6. A computer-readable recording medium for recording a program for causing a computer to function as a three-dimensional panoramic image generation system, a background image inputting step of inputting a plurality of image data serving as a background image actually photographed, A panorama image generation step of combining a plurality of image data input in the background image input step to generate one panorama image, and object data inputting object data composed of three-dimensional graphic data An input step; and a light source information inputting step of inputting, to the object data input in the object data inputting step, light source information having substantially the same light source setting as the image data input in the background image inputting step. And the light source information input step. A combining step of combining the object data input in the object data input step with the panoramic image generated in the panoramic image generating step in accordance with the light source information input by the step, and displaying the panoramic image combined in the combining step A recording medium for storing a program that realizes a display step of performing the following.