JP2004297734A - Electronic mirror system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an electronic mirror system capable of displaying the same image as an image appearing in an actual mirror and processing, arranging and transmitting the image. <P>SOLUTION: The electronic mirror system mainly comprises: a display apparatus 11 being a display means for displaying an imaged image; two cameras 12, 13 being imaging means for imaging a user 14 being an imaging object; and a control section 15 for using the images imaged by the cameras 12, 13 to generate an image of the user 14 facing the front and displaying this synthetic image on the display apparatus 11. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an electronic mirror system that digitizes a captured image and displays it as a mirror on a display device located in front of an object to be imaged.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there is an electronic mirror system that captures a human face and displays the captured image on a display. This system is described, for example, in JP-A-2001-25004. In this conventional electronic mirror system, as shown in FIG. 5, cameras 52 and 53 are installed in front and behind a user 51 who is an object to be imaged, and images taken by the cameras 52 and 53 are displayed on a monitor 54. To be displayed. The operation of these cameras 52 and 53 is performed by a remote controller 55 at the feet of the user 51. As a result, the zoom, elevation and horizontal angles of the cameras 52 and 53 can be controlled. The monitor 54 is divided into two screens, and images captured by the cameras 52 and 53 are individually displayed on the divided screens.
Japanese Patent Application Laid-Open No. 2001-331549 discloses a technique in which a user's face is imaged by a camera, and the size of a frame image is adjusted according to the distance between pupils of the imaged image.
[0003]
[Patent Document]
JP 2001-25004 A [Patent Document]
JP 2001-331549 A
[Problems to be solved by the invention]
However, in the above-described conventional technology, only the image captured by the camera can be displayed, and the captured image cannot be processed, arranged, or further transmitted.
[0005]
The present invention has been made in view of such a point, and provides an electronic mirror system that can perform image display in the same manner as an image projected on an actual mirror, and that can perform image processing, arrangement, and transmission. The purpose is to provide.
[0006]
[Means for Solving the Problems]
An electronic mirror system according to the present invention includes a display unit (display) disposed in front of an object to be imaged, and at least two imaging units installed at positions other than the arrangement position of the display unit and imaging the object to be imaged from different viewpoints. Means (camera); and control means (control unit) for causing the display means to display an image of the object from a viewpoint different from the viewpoint in the imaging from the image captured by the imaging means. And
According to this configuration, images from various viewpoints can be created, and images at various angles can be displayed.
[0007]
In the electronic mirror system according to the present invention, the control means may use the images of the image pickup object at different viewpoints taken by two image pickup means arranged near both sides of the display means, and use the images of the image pickup object facing the front. Is preferably obtained.
According to this configuration, it is possible to display an image in front of the object to be captured without placing the camera in front of the object to be captured, and to function as a mirror.
[0008]
In the electron mirror system according to the present invention, it is preferable that the control unit includes an image processing unit that performs image processing on the captured image and / or the obtained image.
According to this configuration, it is possible to arbitrarily process and arrange an image obtained by using the captured image and / or the captured image.
[0009]
In the electronic mirror system according to the aspect of the invention, it is preferable that the control unit includes a communication control unit (communication control unit) that transmits an image to another device.
According to this configuration, it is possible to freely transmit a captured image or a processed image to another device.
[0010]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
In the present embodiment, an image in which the user faces the front is created using images captured by two cameras installed on both sides of the display means, and the obtained image is displayed on the display means. The following describes a case in which this is performed. The number of cameras is not limited to two, and may be three or more. The position where the cameras are installed is not limited to both sides of the display unit.
[0011]
FIG. 1 is a diagram showing a schematic configuration of an electron mirror system according to an embodiment of the present invention. The system illustrated in FIG. 1 includes a display 11 that is a display unit that displays a captured image, two cameras 12 and 13 that are an imaging unit that captures an image of a user 14 that is an object to be captured, and images captured by the cameras 12 and 13. The control unit 15 mainly generates an image in which the user 14 faces the front using the image thus obtained, and displays the image on the display 11.
[0012]
Examples of the display 11 include light-emitting and light-receiving display devices. For example, a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display) device, a PDP (Plasma Display Panel), an EL (Electro Luminescence) device, a conductive polymer display device, and the like can be given.
[0013]
The camera 12 includes, for example, a CCD (Charge Coupled Diode) camera, but is not particularly limited to a CCD camera as long as a captured image can be digitized.
[0014]
The control unit 15 has a configuration shown in FIG. 2 in detail. FIG. 2 is a block diagram showing the internal configuration of the control unit 15 of the electronic mirror system shown in FIG. The control unit 15 includes an image acquisition unit 151 that captures images captured by the cameras 12 and 13, an image processing unit 152 that performs predetermined image processing on the captured images, and an image storage unit that stores the captured images. 153, a display control unit 154 that controls display corresponding to an image to be displayed, and a communication control unit 155 that transmits and receives an image to and from another device.
In the control unit 15, the images captured by the cameras 12 and 13 are sent to the image acquisition unit 151 and sent to the image processing unit 152 as they are, or sent to the image storage unit 153 and stored. The image storage unit 153 stores the image acquired by the image acquisition unit 151, and outputs the image to the image processing unit 152 as needed.
[0015]
The image processing unit 152 performs predetermined image processing on the image sent from the image storage unit 153 or the image acquisition unit 151. The image processing unit 152 includes a configuration for creating an image as shown in FIG. This configuration for creating an image includes a feature point extraction unit 1521 that extracts feature points in a plurality of images, analyzes a relationship between feature points extracted from each image, and recognizes a three-dimensional shape based on the analysis result. And a calculation unit 1523 that creates a target image based on the recognized three-dimensional shape.
In the image processing unit 152, for example, a process of adjusting the color of the image of the object to be captured to an arbitrary color, a process of inverting the obtained image in the horizontal direction, and a fictitious item (for example, glasses , Accessories, and costumes) can be appropriately performed. These image processes can be performed, for example, by operating the operation units 11a and 11b of the display 11. The image processed by the image processing unit 152 or the image obtained by the image obtaining unit 151 is output to the display control unit 154 and is output to the communication control unit 155 as necessary.
[0016]
The display control unit 154 controls the driving of the display 11 according to the image output from the image processing unit 152. That is, drive control of each pixel of the display 11 is performed based on an image to be displayed.
[0017]
The communication control unit 155 performs control of transmitting an image processed by the image processing unit 152, an image captured by the cameras 12, 13 or an image processed and / or created by the image processing unit 152 to another device. . For example, when an image is output from a printer, the image is transmitted to the printer, and when the image is transmitted to another location, the image is transmitted via a line. The transmission form may be wired or wireless, or may be infrared or Bluetooth.
[0018]
Next, the operation of the electronic mirror system having the above configuration will be described.
The user 14 looks at the display 11 arranged in front of the user 14. At this time, the user 14 is imaged by the two cameras 12 and 13 arranged near both sides of the display 11. That is, the camera 12 images the user 14 from the left side as viewed from the user 14, and the camera 13 images the user 14 from the right side as viewed from the user 14. As shown in FIG. 1, the viewpoint of the camera 12 is different from the viewpoint of the camera 13. Therefore, as shown in FIG. 4, an image captured by the camera 12 (FIG. 4B) and an image captured by the camera 13 (FIG. 4 (a)) is different.
The images captured by the cameras 12 and 13 are acquired by the image acquisition unit 151 of the control unit 15 and stored in the image storage unit 153. The image stored in the image storage unit 153 is subjected to predetermined image processing in the image processing unit 152 as needed.
[0019]
The image processing unit 152 first creates an image in which the user 14 faces the front from the image (image 1) captured by the camera 12 and the image (image 2) captured by the camera 13. The image processing unit 152 extracts an image captured by the camera 12 and an image captured by the camera 13 stored in the image storage unit 152. Then, view morphing is performed on these two images. This view morphing is a technique for generating an object image at an arbitrary viewpoint position between images based on a plurality of object images having different viewpoint positions. This technique is described in "View Morphing" 1996 SIGGRAPH Steven M. This is a technique disclosed in Seitz (CMU) and the like. All this content is included here.
When performing the view morphing process in the image processing unit 152, the feature points in the image 1 and the image 2 are extracted in the feature point extracting unit 1521. As this feature point, for example, the highest part in the face such as the head of the nose is selected. The feature point is the same part in the image 1 and the image 2.
[0020]
After the feature points are extracted in this way, the three-dimensional shape recognition unit 1522 recognizes the three-dimensional shape of the user 14. Specifically, the image 1 and the image 2 are compared, and the relative position change amount between them is obtained. Since the relative position change amount is related to the distance between the user 14 and the cameras 12 and 13, the three-dimensional shape is recognized by performing a geometric calculation process based on the relative position change amount. Based on the three-dimensional shape, the calculation unit 1523 creates an image. In this way, the view morphing process appropriately reproduces a three-dimensional spatial image without using depth information.
[0021]
The image created as described above is sent to the display control unit 154 and stored in the image storage unit 153. The display control unit 154 performs pixel control according to the created image, and causes the display 11 to display the image.
In this manner, an image viewed from a viewpoint different from the viewpoints of the cameras 12 and 13 can be created and displayed. That is, as shown in FIGS. 1 and 4, a face image viewed from the front (FIG. 4A) and a face image viewed from the left (FIG. 4 (c)) can be created and displayed.
[0022]
In order to create an image that is as natural as possible, for example, a point that moves up, down, left, and right is displayed on the screen when the program starts, and the movement of that point is followed, and the user at that time is imaged with a camera. The image may be obtained in advance, and the image may be created using the image as appropriate. Thereby, a more natural image can be obtained.
[0023]
According to the present embodiment, images at various viewpoints can be created, images at various angles can be displayed, and even if the camera is not arranged in front of the object, It is possible to display an image of the front and function as a mirror.
[0024]
The image of the user 14 facing the front, which is created by the image processing unit 152, is displayed on the display 11, and when the user 14 processes and arranges the image, for example, the operation unit 11a of the display 11, 11b is used to instruct image processing. The image processing instruction may be performed using the operation units 11a and 11b of the display 11, or may be performed by key input using a keyboard, dragging using a mouse, or pen input using an electronic pen.
[0025]
This instruction is sent to the image processing unit 152, and the image processing unit 152 extracts an image captured from the image storage unit 153 and performs predetermined image processing on the image. More specifically, the image processing includes changing the face color and hair color of the image to an arbitrary color, adding imaginary items such as glasses, accessories, and costumes to the image, and storing the image in the image storage unit 153 in advance. The process of synthesizing this image with other images (such as people and backgrounds) stored in the image, such as the face parts (eyes, nose, mouth, ears, contours, etc.) and body parts (hands, Processing for deforming the legs, chest, belly, etc.).
[0026]
The image processed in this way is displayed on the display 11 under the control of the display control unit 154. The user 14 may further instruct image processing while viewing this image, or may transmit the image processing to another device as needed. When transmitting an image to another device, the communication control unit 155 transmits the image to another device via a line. For example, the image may be transmitted to a printer and printed out, or may be transmitted to another device (a server, a personal computer, or the like) via a network such as the Internet. This makes it possible to freely transmit a captured image or a processed image to another device.
[0027]
As described above, the electronic mirror system according to the present embodiment can perform image display in the same manner as an image projected on an actual mirror using images captured from different viewpoints, and furthermore, image processing, arrangement, Transmission can take place.
[0028]
With such a system, for example, it is also possible to transmit and receive an image to and from a cosmetics company to receive makeup guidance and consult skin troubles. In addition, it becomes easy to try on and match costumes on the Internet shopping. Further, by combining with a background image, a travel photograph or the like can be created. It is also possible to perform human divination using image analysis software.
[0029]
The present invention is not limited to the above embodiment, but can be implemented with various modifications. In the above embodiment, the case where the image is a still image is described. However, the present invention can be applied to a case where the image is a moving image.
[0030]
Further, in the above-described embodiment, a case is described in which a front-facing image is created using images captured from left and right viewpoints. In the present invention, a new virtual image is created from a plurality of images at different viewpoints. The present invention can be applied to general cases of creating an image. For example, by installing a camera behind the user and also using an image taken by the camera, an image of the rear view of the user can be created, thereby making it possible to check a hairstyle and the like. Become.
[0031]
【The invention's effect】
As described above, the electronic mirror system of the present invention can perform image display using images captured from different viewpoints in the same manner as an image projected on an actual mirror. Further, by providing the image processing means, it is possible to process, arrange, and transmit images.
[Brief description of the drawings]
FIG. 1 is a diagram showing a schematic configuration of an electron mirror system according to an embodiment of the present invention.
FIG. 2 is a block diagram showing an internal configuration of a control unit of the electronic mirror system shown in FIG.
FIG. 3 is a diagram for explaining a processing unit included in the image processing unit shown in FIG. 2;
FIG. 4 is a diagram showing an image captured and synthesized by the electron mirror system according to the embodiment of the present invention.
FIG. 5 is a diagram showing a schematic configuration of a conventional electron mirror system.
[Explanation of symbols]
11 Display 11a, 11b Operation Unit 12, 13 Camera 14 User 15 Control Unit 151 Image Acquisition Unit 152 Image Processing Unit 153 Image Storage Unit 154 Display Control Unit 155 Communication Control Unit 1521 Feature Point Extraction Unit 1522 3D Shape Recognition Unit 1523 Operation Unit

Claims (4)

Display means arranged in front of the object to be imaged;
At least two image pickup units installed at positions other than the arrangement position of the display unit, and imaging the object to be imaged from different viewpoints,
An electronic mirror system comprising: a control unit that causes the display unit to display an image of the object to be captured from a viewpoint different from a viewpoint in the image from the images captured by the imaging unit. The control means obtains an image of the object facing forward using images of the object at different viewpoints imaged by two imagers arranged near both sides of the display means. The electronic mirror system according to claim 1, wherein The electron mirror system according to claim 1, wherein the control unit includes an image processing unit that performs image processing on a captured image and / or an obtained image. 4. The electron mirror system according to claim 1, wherein the control unit includes a communication control unit that transmits an image to another device.

Images