JP4145121B2 - Image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention uses the image for the left eye (left image) and a right-eye image (right image), relates to an image processing apparatus for displaying a stereoscopic image.
[0002]
[Prior art]
There exists a thing of patent document 1 as this kind of conventional technology. Here, the image of the left subject and the background image are combined to form the left composite image, and the image of the right subject and the background image are combined to form the right composite image. And the stereoscopic image is displayed using the composite image on the right, and the subject image when the stereoscopic image is displayed by adjusting the synthesis position of the subject image and the background image according to the shooting distance of the subject, and The contradiction in the positional relationship of the background image is resolved.
[0003]
[Patent Document 1]
JP-A-5-346778 [0004]
[Problems to be solved by the invention]
However, in Patent Document 1, even if a subject image and a background image are combined, another image is combined with the subject image itself, the subject image is processed, and the processed subject is stereoscopically displayed. Such a technique was desired.
[0005]
Therefore, the present invention has been made in view of the above-described conventional problems, and when a stereoscopic image is displayed using the left image and the right image, another image is synthesized with the subject image itself. Another object of the present invention is to provide a recording medium on which an image processing apparatus capable of modifying or supplementing an image of a subject is recorded.
[0006]
[Means for Solving the Problems]
In order to solve the above-described conventional problems, an image processing apparatus of the present invention includes a projection image generation unit that generates left and right projection images when an original subject is viewed from two viewpoints or two directions, and left and right template images. The template image storage means for storing and the left projection image and the left template image are combined to form a left composite image when the processed subject is viewed from one viewpoint or direction, and the right projection Image synthesizing means for synthesizing the image and the right template image and forming a right synthesized image when the processed subject is viewed from the other viewpoint or direction.
[0007]
According to the present invention having such a configuration, left and right projection images representing the original subject are generated, the left projection image and the left template image are synthesized to form a left synthesized image, and the right projection image is created. The right template image is synthesized to form the right synthesized image. That is, the left and right composite images are formed by processing the projection image showing the subject with the template image. Accordingly, when a stereoscopic image is displayed using the left and right composite images, the processed subject can be stereoscopically viewed.
[0008]
If various template images are prepared, a projected image of a subject can be combined with each template image. Furthermore, the template image can be reused repeatedly, and photographing and calculation processing for generating the template image can be reduced.
[0009]
On the other hand, the image processing apparatus of the present invention includes projection image generation means for generating left and right projection images when the original subject is viewed from two viewpoints or two directions, a template image, and depth information indicating the depth of the template image. The storage means for storing, the template image generation means for generating the left and right template images using the template image and depth information in the storage means, the left projection image and the left template image are synthesized and processed When the subject is viewed from one viewpoint or direction, the left composite image is formed, and the right projection image and right template image are combined, and the processed subject is viewed from the other viewpoint or direction Image synthesizing means for forming a right synthesized image.
[0010]
Here, the template image and depth information indicating the depth of the template image are stored, and the left and right template images are generated using the template image and the depth information. For this reason, the depth of the stereoscopic image represented by the left and right template images can be changed by changing the depth information. Therefore, it is not necessary to store template images having different depths from each other, and the storage capacity can be reduced.
[0011]
Then, the left projection image and the left template image are synthesized to form a left synthesized image, and the right projection image and the right template image are synthesized to form a right synthesized image. Accordingly, when a stereoscopic image is displayed using the left and right composite images, the processed subject can be stereoscopically viewed.
[0012]
The template image is preferably indicated by hierarchical image data or vector graphics data.
[0013]
These data make it possible to reproduce a high-quality template image regardless of the enlargement / reduction of the template image. In addition, the amount of data is small, and the storage capacity can be saved.
[0014]
In the present invention, the image synthesizing unit synthesizes the projection image and the template image after subjecting at least a part of each image to transmission processing to form a synthesized image.
[0015]
By such a transmission process, it is possible to flexibly combine the projection image and the template image.
[0016]
Further, in the present invention, the template image includes a change in light and shade, and the image composition means performs a transparent process on the light and shade change in the template image, and then combines the projection image and the template image to form a composite image. ing.
[0017]
In this case, by combining the template image with the projection image, it is possible to add light and shade to the combined image.
[0018]
In the present invention, the original subject is a face. Alternatively, the template image includes at least one of a face outline and a hair portion.
[0019]
In this case, the projected image of the face and the template image of the face outline and hair portion are synthesized, and the processed face of the subject can be stereoscopically viewed.
[0027]
DETAILED DESCRIPTION OF THE INVENTION
First, the outline of the present invention will be described.
[0028]
In a three-dimensional display, a left image and a right image are created in consideration of the parallax between the left and right eyes of the viewer, so-called binocular parallax, and the subject is stereoscopically viewed by viewing these images with the left and right eyes substantially simultaneously. Show me. The left image and the right image are created by photographing the subject from two directions, left and right, using two cameras, or by creating a three-dimensional subject using computer graphics. The object is created by looking at the subject from the left and right directions. Alternatively, the left image and the right image may be created by synthesizing an image photographed by the camera and an image created by computer graphics.
[0029]
In the present invention, the left image and the right image are used to not only show the subject stereoscopically, but also the left and right projection images indicating the original subject are combined with the left and right template images to show the original subject. The projection image is modified or supplemented by the template image, and the modified or supplemented subject is shown in three dimensions.
[0030]
For example, if the original subject is a human face and the left and right template images are shade patterns indicating the shadows of the faces, the left and right projected images representing the subject are combined with the left and right template images to create a shadowed face. Show three-dimensionally. Alternatively, if the left and right template images show the contours of the face, the left and right projected images showing the subject are combined with the left and right template images, and the face whose contour has been changed is viewed three-dimensionally. In addition, the left and right template images are prepared with shading patterns showing various face shadows and various face outlines, and the face is modified or supplemented by selectively applying the left and right template images. And show it in three dimensions.
[0031]
Next, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0032]
FIG. 1 is a block diagram showing a first embodiment of the image processing apparatus of the present invention. The image processing apparatus 10 of this embodiment includes a projection image generation unit 11 that generates left and right projection images when the original subject is viewed from two viewpoints or two directions, and a template image storage unit 12 that stores left and right template images. The left projection image and the left template image are combined to form the left composite image, and the right projection image and the right template image are combined to form the right composite image. And.
[0033]
FIG. 2 is a flowchart showing a processing process of the image processing apparatus 10. The processing steps of the image processing apparatus 10 will be sequentially described according to this flowchart.
[0034]
First, the projection image generation unit 11 generates right and left projection images when the original subject is viewed from two viewpoints or two directions (step S101).
[0035]
For example, as shown in FIG. 3, the left and right projected images are generated by projecting the subject 21 onto the image planes 22 and 23 at the respective viewpoints separated from each other in the left and right directions. Actually, the subject 21 is photographed by two cameras separated from each other on the left and right, or the subject 21 is photographed by one camera and a stereo adapter to generate left and right projection images.
[0036]
Alternatively, computer graphics software can be executed by a personal computer or the like to generate left and right projection images. When using computer graphics, a perspective projection method as shown in FIG. 4A or an approximate parallel projection method as shown in FIG. 4B is used. In the perspective projection method of FIG. 4A, two virtual cameras are arranged on a straight line with an appropriate distance left and right, and the subject 21 is photographed by each virtual camera to generate left and right projection images. Further, in the parallel projection method of FIG. 4B, two virtual cameras are placed at an appropriate distance on the left and right sides, the virtual cameras are directed to the subject 21 at an appropriate angle, and the virtual cameras are positioned between the virtual cameras. An angle difference is set, and the subject 21 is photographed by each virtual camera to generate left and right projection images.
[0037]
In the case of the perspective projection method, the farther the subject 21 is from the lens surface of each virtual camera, the smaller the left and right projection images, and the right and left projection images approximate to those taken by two actual cameras. An arithmetic process of dividing the left and right projection images by the distance between the subject 21 and the lens surface of each virtual camera is required.
[0038]
In the case of the parallel projection method, the sizes of the left and right projection images do not change according to the distance between the subject 21 and the lens surface of each virtual camera, and an arithmetic process like the perspective projection method is not required. In particular, if the depth of the subject 21 is short, a large error does not occur in the left and right projection images, and the cost for the arithmetic processing can be reduced. However, if the distance a or the angle difference α between the virtual cameras as shown in FIG. 5 is too small, the stereoscopic effect of the subject 21 when the left and right projected images are viewed substantially simultaneously with the left and right eyes becomes poor. If the distance a or the angle difference α between the virtual cameras is too large, it is difficult to take a left-right correspondence when the left and right projected images are viewed substantially simultaneously with the left and right eyes, and stereoscopic viewing itself becomes difficult. Although it depends on the specification of the three-dimensional display, it is generally desirable to set the angle difference α in the range of 3 to 15 degrees.
[0039]
Next, the image composition unit 13 receives the left and right projection images from the projection image generation unit 11, reads the left and right template images from the template image storage unit 12, and synthesizes the left projection image and the left template image, In addition to forming the left composite image, the right projection image and the right template image are combined to form the right composite image (step S102). That is, the left and right projection images representing the original subject 21 are combined with the left and right template images, and the left and right projection images are modified or corrected to form left and right synthesized images representing the modified or supplemented subject.
[0040]
For example, the left and right template images 24a and 24b as shown in FIG. 6A and the left and right projection images 25a and 25b of the subject as shown in FIG. 6B are synthesized, as shown in FIG. 6C. The right and left composite images 26a and 26b of the subject that have been appropriately modified or supplemented are formed.
[0041]
Further, after applying the transparency process to the template image 27 having the shadow pattern as shown in FIG. 7A, the template image 27 and the projected image 28 of the face as shown in FIG. As shown in (c), a modified or supplemented composite image 29 of the face is formed. Here, the template image 27 of the shadow pattern and the projected image 28 of the face are combined to form a combined image 29 of the face having a shadow.
[0042]
Alternatively, the template images 31a, 31b, 31c, 31d having the contour pattern as shown in FIG. 8A and the central portion (the inner portion of the dotted line) of the face projection image 32 as shown in FIG. 8B are synthesized. Then, the composite images 33a, 33b, 33c, and 33d of the face whose contours are corrected or supplemented as shown in FIG. 8C are formed. Thereby, the outline of a face can be changed variously.
[0043]
Also, each template image 34a, 34b, 34c, 34d of the hairstyle as shown in FIG. 9A and the projected image 35 of the face as shown in FIG. 9B are synthesized and shown in FIG. 9C. The composite images 36a, 36b, 36c, 36d of the face modified or supplemented with various hairstyles are formed. Thereby, the hairstyle of a face can be changed variously.
[0044]
In FIG. 7, FIG. 8, and FIG. 9, the left and right images are not shown separately, and only one image is shown for simplification. In addition to FIGS. 6 to 9, projection images and template images of various subjects can be assumed.
[0045]
When the left and right composite images are formed in this way and the left and right composite images are viewed substantially simultaneously with the left and right eyes using a three-dimensional display, the corrected or supplemented subject can be viewed in three dimensions.
[0046]
As a three-dimensional display, a head-mounted display having respective liquid crystal display panels for displaying left and right composite images, a display of a method for alternately displaying left and right composite images in a time-division manner using liquid crystal glasses, and a method using a lenticular lens There are other types of displays as long as they can see the composite image of the left and right eyes with the left and right eyes and view the subject in three dimensions. .
[0047]
FIG. 10 is a block diagram showing a second embodiment of the image processing apparatus of the present invention. In FIG. 10, the same reference numerals are given to parts that perform the same function as the apparatus of FIG. 1.
[0048]
An image processing apparatus 10A according to the present embodiment includes a template storage unit with depth 41 that stores a template image and depth information indicating the depth of the template image, and a template with depth instead of the template image storage unit 12 in the apparatus of FIG. A template image generation unit 42 that generates left and right template images using the template image and depth information in the storage unit 41 is applied.
[0049]
FIG. 11 is a flowchart showing the processing steps of the image processing apparatus 10A. The processing steps of the image processing apparatus 10A will be described sequentially according to this flowchart.
[0050]
First, the projection image generation unit 11 generates right and left projection images when the original subject is viewed from two viewpoints or two directions (step S201). The process in step S201 is the same as the process in step S101 in FIG.
[0051]
Next, the template image generation unit 42 reads the template image and the depth information from the template storage unit 41 with depth, and generates left and right template images using these (Step S202).
[0052]
For example, the template image 51 including the objects 51a and 51b as shown in FIG. 12A and the depth information 52a and 52b of the objects 51a and 51b are read from the template storage unit 41 with depth and shown in FIG. Similarly, the left and right template images 53 and 54 are generated in which the object 51a is positioned according to the depth indicated by the depth information 52a and the object 51b is positioned according to the depth indicated by the depth information 52b.
[0053]
Generally, an object is positioned to the right in the left image, and the object is positioned to the left in the right image, so that the object can be viewed stereoscopically when the left and right images are viewed substantially simultaneously. In addition, the shorter the distance of the object, the longer the distance that the object moves to the left and right (the distance that the object moves is inversely proportional to the distance of the depth).
[0054]
In the template image 51 of FIG. 12A, the value of the depth information 52a of the object 51a is “1” and the value of the depth information 52b of the object 51b is “3”. The distance becomes longer. The value “3” of the depth information 52b indicates a distance that is three times as deep as the value “1” of the depth information 52a. For this reason, in the left and right template images 53 and 54 in FIG. 12B, the moving distance of the object 51a is longer than the moving distance of the object 51b.
[0055]
Moreover, it is preferable that the template image 51 is shown by image data with a hierarchy or vector graphics data. These data make it possible to reproduce a high-quality template image regardless of the enlargement / reduction of the template image 51. Further, the amount of data is small, and the capacity of the template storage unit 41 with depth can be saved.
[0056]
Next, the image synthesis unit 13 receives the left and right projection images from the projection image generation unit 11, receives the left and right template images from the template image generation unit 42, synthesizes the left projection image and the left template image, And the right projection image and the right template image are combined to form the right composite image (step S203).
[0057]
When the left and right composite images are formed in this way and the left and right composite images are viewed substantially simultaneously with the left and right eyes using a three-dimensional display, the corrected or supplemented subject can be viewed in three dimensions.
[0058]
Various projection images and template images of the subject can be assumed as in the first embodiment.
[0059]
The present invention is achieved by executing a program for realizing the image processing method by hardware such as a CPU or a computer .
[0060]
The program is recorded on a recording medium such as a magnetic disk, an optical disk, or a hard disk built in a computer, or transmitted and received through a communication network. A terminal device or the like with a computer or a microprocessor can implement the present invention by reading a program from a recording medium or receiving the program through a communication network such as the Internet and executing the program. When the program is received through a communication network such as the Internet, a download program is required. The download program is also recorded on a recording medium such as a magnetic disk, an optical disk, or a hard disk built in the computer, or transmitted and received through a communication network. Also, these programs can be periodically downloaded to the terminal device via the Internet.
[0061]
Furthermore, in a system including a terminal device such as a computer and a network, a plurality of processes can be distributed to the terminal devices such as a computer. Therefore, the program can be applied not only to a single terminal device such as a computer but also to a system. In this system, a television, a video camera, a personal computer, a CRT, a liquid crystal display, a printer, a server, and the like are connected through a network. Examples of the network include the Internet and a LAN.
[0062]
Examples of the recording medium include semiconductor memory elements such as mask ROM, EPROM, EEPROM, and flash ROM, IC cards, hard disks, flexible disks, optical disks such as MO, CD, MD, and DVD, magnetic tapes, and the like. Other types of recording media may be used as long as they can be recorded.
[0063]
【The invention's effect】
As described above, according to the present invention, left and right projection images showing the original subject are generated, the left projection image and the left template image are synthesized to form a left synthesized image, and the right projection image is created. The right template image is synthesized to form the right synthesized image. That is, the left and right composite images are formed by processing the projection image showing the subject with the template image. Accordingly, when a stereoscopic image is displayed using the left and right composite images, the processed subject can be stereoscopically viewed.
[0064]
If various template images are prepared, a projected image of a subject can be combined with each template image. Furthermore, the template image can be reused repeatedly, and photographing and calculation processing for generating the template image can be reduced.
[0065]
Further, the template image and depth information indicating the depth of the template image are stored, and the left and right template images are generated using the template image and the depth information. For this reason, the depth of the stereoscopic image represented by the left and right template images can be changed by changing the depth information. Therefore, it is not necessary to store template images having different depths from each other, and the storage capacity can be reduced.
[0066]
Furthermore, since the template image is stored as hierarchical image data or vector graphics data, a high-quality template image can be reproduced regardless of the enlargement / reduction of the template image. In addition, the amount of data is small, and the storage capacity can be saved.
[0067]
Further, since at least a part of the image is subjected to the transmission process and then combined to form the combined image, the projection image and the template image can be combined flexibly.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a first embodiment of an image processing apparatus according to the present invention.
FIG. 2 is a flowchart showing a processing process of the image processing apparatus of FIG. 1;
FIG. 3 is a diagram illustrating a method for generating left and right projection images.
4A is a diagram showing a perspective projection method for generating left and right projection images, and FIG. 4B is a diagram showing a parallel projection method for generating left and right projection images.
FIG. 5 is a diagram showing the parallel projection method for generating left and right projection images in more detail.
6A, 6B, and 6C are diagrams illustrating examples of left and right template images, left and right projection images, and left and right composite images.
7A, 7B, and 7C are diagrams illustrating other examples of a template image, a projection image, and a composite image.
FIGS. 8A, 8B, and 8C are diagrams illustrating another example of a template image, a projection image, and a composite image.
FIGS. 9A, 9B, and 9C are diagrams illustrating still another example of a template image, a projection image, and a composite image.
FIG. 10 is a block diagram showing a second embodiment of the image processing apparatus of the present invention.
11 is a flowchart showing a processing process of the image processing apparatus of FIG.
12A is a diagram illustrating template images and depth information, and FIG. 12B is a diagram illustrating left and right template images.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10,10A Image processing apparatus 11 Projection image generation part 12 Template image storage part 13 Image composition part 21 Subject 41 Template storage part 42 with depth Template image generation part

Claims (5)

Projection image generation means for generating left and right projection images when the original subject is viewed from two viewpoints or two directions;
Template image storage means for storing left and right template images indicating the shadow pattern or contour pattern of the subject;
The left template image is synthesized with the left projection image, and the left projection image is corrected or supplemented to form a left synthesized image when the subject is viewed from one viewpoint or direction, and the right projection image A right template image, correcting or supplementing the right projection image, and forming a right composite image when the subject is viewed from the other viewpoint or direction, with a shadow or contour An image processing apparatus for forming a left and right composite image to which is assigned. Projection image generation means for generating left and right projection images when the original subject is viewed from two viewpoints or two directions;
Storage means for storing a template image indicating the shadow pattern or contour pattern of the subject and depth information indicating the depth of the template image;
Template image generation means for generating left and right template images using the template image and depth information in the storage means;
The left template image is synthesized with the left projection image, and the left projection image is corrected or supplemented to form a left synthesized image when the subject is viewed from one viewpoint or direction, and the right projection image A right template image, correcting or supplementing the right projection image, and forming a right composite image when the subject is viewed from the other viewpoint or direction, with a shadow or contour An image processing apparatus for forming a left and right composite image to which is assigned. The image processing apparatus according to claim 2, wherein the template image in the storage unit is indicated by hierarchical image data or vector graphics data.   3. The image processing apparatus according to claim 1, wherein the image synthesizing unit forms the synthesized image by synthesizing the projection image and the template image after transmitting at least a part of each image.   The image processing apparatus according to claim 1, wherein the original subject is a face.

Images