JP2004297734A - Electronic mirror system - Google Patents

Electronic mirror system Download PDF

Info

Publication number
JP2004297734A
JP2004297734A JP2003091040A JP2003091040A JP2004297734A JP 2004297734 A JP2004297734 A JP 2004297734A JP 2003091040 A JP2003091040 A JP 2003091040A JP 2003091040 A JP2003091040 A JP 2003091040A JP 2004297734 A JP2004297734 A JP 2004297734A
Authority
JP
Japan
Prior art keywords
image
display
captured
unit
control unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2003091040A
Other languages
Japanese (ja)
Inventor
Koji Yoshioka
紘二 吉岡
Original Assignee
Aruze Corp
Sus 21:Kk
アルゼ株式会社
株式会社エス・ユー・エス・二十一
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aruze Corp, Sus 21:Kk, アルゼ株式会社, 株式会社エス・ユー・エス・二十一 filed Critical Aruze Corp
Priority to JP2003091040A priority Critical patent/JP2004297734A/en
Publication of JP2004297734A publication Critical patent/JP2004297734A/en
Application status is Pending legal-status Critical

Links

Images

Abstract

An electronic mirror system capable of displaying an image in the same manner as an image projected on an actual mirror, and further capable of processing, arranging, and transmitting the image.
A display 11 is a display unit that displays a captured image, two cameras 12 and 13 are an imaging unit that captures a user 14 that is an object to be imaged, and an image captured by the cameras 12 and 13 is displayed. The control unit 15 mainly generates an image in which the user 14 faces the front, and displays the composite image on the display 11.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an electronic mirror system that digitizes a captured image and displays it as a mirror on a display device located in front of an object to be imaged.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, there is an electronic mirror system that captures a human face and displays the captured image on a display. This system is described, for example, in JP-A-2001-25004. In this conventional electronic mirror system, as shown in FIG. 5, cameras 52 and 53 are installed in front and behind a user 51 who is an object to be imaged, and images taken by the cameras 52 and 53 are displayed on a monitor 54. To be displayed. The operation of these cameras 52 and 53 is performed by a remote controller 55 at the feet of the user 51. As a result, the zoom, elevation and horizontal angles of the cameras 52 and 53 can be controlled. The monitor 54 is divided into two screens, and images captured by the cameras 52 and 53 are individually displayed on the divided screens.
Japanese Patent Application Laid-Open No. 2001-331549 discloses a technique in which a user's face is imaged by a camera, and the size of a frame image is adjusted according to the distance between pupils of the imaged image.
[0003]
[Patent Document]
JP 2001-25004 A [Patent Document]
JP 2001-331549 A
[Problems to be solved by the invention]
However, in the above-described conventional technology, only the image captured by the camera can be displayed, and the captured image cannot be processed, arranged, or further transmitted.
[0005]
The present invention has been made in view of such a point, and provides an electronic mirror system that can perform image display in the same manner as an image projected on an actual mirror, and that can perform image processing, arrangement, and transmission. The purpose is to provide.
[0006]
[Means for Solving the Problems]
An electronic mirror system according to the present invention includes a display unit (display) disposed in front of an object to be imaged, and at least two imaging units installed at positions other than the arrangement position of the display unit and imaging the object to be imaged from different viewpoints. Means (camera); and control means (control unit) for causing the display means to display an image of the object from a viewpoint different from the viewpoint in the imaging from the image captured by the imaging means. And
According to this configuration, images from various viewpoints can be created, and images at various angles can be displayed.
[0007]
In the electronic mirror system according to the present invention, the control means may use the images of the image pickup object at different viewpoints taken by two image pickup means arranged near both sides of the display means, and use the images of the image pickup object facing the front. Is preferably obtained.
According to this configuration, it is possible to display an image in front of the object to be captured without placing the camera in front of the object to be captured, and to function as a mirror.
[0008]
In the electron mirror system according to the present invention, it is preferable that the control unit includes an image processing unit that performs image processing on the captured image and / or the obtained image.
According to this configuration, it is possible to arbitrarily process and arrange an image obtained by using the captured image and / or the captured image.
[0009]
In the electronic mirror system according to the aspect of the invention, it is preferable that the control unit includes a communication control unit (communication control unit) that transmits an image to another device.
According to this configuration, it is possible to freely transmit a captured image or a processed image to another device.
[0010]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
In the present embodiment, an image in which the user faces the front is created using images captured by two cameras installed on both sides of the display means, and the obtained image is displayed on the display means. The following describes a case in which this is performed. The number of cameras is not limited to two, and may be three or more. The position where the cameras are installed is not limited to both sides of the display unit.
[0011]
FIG. 1 is a diagram showing a schematic configuration of an electron mirror system according to an embodiment of the present invention. The system illustrated in FIG. 1 includes a display 11 that is a display unit that displays a captured image, two cameras 12 and 13 that are an imaging unit that captures an image of a user 14 that is an object to be captured, and images captured by the cameras 12 and 13. The control unit 15 mainly generates an image in which the user 14 faces the front using the image thus obtained, and displays the image on the display 11.
[0012]
Examples of the display 11 include light-emitting and light-receiving display devices. For example, a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display) device, a PDP (Plasma Display Panel), an EL (Electro Luminescence) device, a conductive polymer display device, and the like can be given.
[0013]
The camera 12 includes, for example, a CCD (Charge Coupled Diode) camera, but is not particularly limited to a CCD camera as long as a captured image can be digitized.
[0014]
The control unit 15 has a configuration shown in FIG. 2 in detail. FIG. 2 is a block diagram showing the internal configuration of the control unit 15 of the electronic mirror system shown in FIG. The control unit 15 includes an image acquisition unit 151 that captures images captured by the cameras 12 and 13, an image processing unit 152 that performs predetermined image processing on the captured images, and an image storage unit that stores the captured images. 153, a display control unit 154 that controls display corresponding to an image to be displayed, and a communication control unit 155 that transmits and receives an image to and from another device.
In the control unit 15, the images captured by the cameras 12 and 13 are sent to the image acquisition unit 151 and sent to the image processing unit 152 as they are, or sent to the image storage unit 153 and stored. The image storage unit 153 stores the image acquired by the image acquisition unit 151, and outputs the image to the image processing unit 152 as needed.
[0015]
The image processing unit 152 performs predetermined image processing on the image sent from the image storage unit 153 or the image acquisition unit 151. The image processing unit 152 includes a configuration for creating an image as shown in FIG. This configuration for creating an image includes a feature point extraction unit 1521 that extracts feature points in a plurality of images, analyzes a relationship between feature points extracted from each image, and recognizes a three-dimensional shape based on the analysis result. And a calculation unit 1523 that creates a target image based on the recognized three-dimensional shape.
In the image processing unit 152, for example, a process of adjusting the color of the image of the object to be captured to an arbitrary color, a process of inverting the obtained image in the horizontal direction, and a fictitious item (for example, glasses , Accessories, and costumes) can be appropriately performed. These image processes can be performed, for example, by operating the operation units 11a and 11b of the display 11. The image processed by the image processing unit 152 or the image obtained by the image obtaining unit 151 is output to the display control unit 154 and is output to the communication control unit 155 as necessary.
[0016]
The display control unit 154 controls the driving of the display 11 according to the image output from the image processing unit 152. That is, drive control of each pixel of the display 11 is performed based on an image to be displayed.
[0017]
The communication control unit 155 performs control of transmitting an image processed by the image processing unit 152, an image captured by the cameras 12, 13 or an image processed and / or created by the image processing unit 152 to another device. . For example, when an image is output from a printer, the image is transmitted to the printer, and when the image is transmitted to another location, the image is transmitted via a line. The transmission form may be wired or wireless, or may be infrared or Bluetooth.
[0018]
Next, the operation of the electronic mirror system having the above configuration will be described.
The user 14 looks at the display 11 arranged in front of the user 14. At this time, the user 14 is imaged by the two cameras 12 and 13 arranged near both sides of the display 11. That is, the camera 12 images the user 14 from the left side as viewed from the user 14, and the camera 13 images the user 14 from the right side as viewed from the user 14. As shown in FIG. 1, the viewpoint of the camera 12 is different from the viewpoint of the camera 13. Therefore, as shown in FIG. 4, an image captured by the camera 12 (FIG. 4B) and an image captured by the camera 13 (FIG. 4 (a)) is different.
The images captured by the cameras 12 and 13 are acquired by the image acquisition unit 151 of the control unit 15 and stored in the image storage unit 153. The image stored in the image storage unit 153 is subjected to predetermined image processing in the image processing unit 152 as needed.
[0019]
The image processing unit 152 first creates an image in which the user 14 faces the front from the image (image 1) captured by the camera 12 and the image (image 2) captured by the camera 13. The image processing unit 152 extracts an image captured by the camera 12 and an image captured by the camera 13 stored in the image storage unit 152. Then, view morphing is performed on these two images. This view morphing is a technique for generating an object image at an arbitrary viewpoint position between images based on a plurality of object images having different viewpoint positions. This technique is described in "View Morphing" 1996 SIGGRAPH Steven M. This is a technique disclosed in Seitz (CMU) and the like. All this content is included here.
When performing the view morphing process in the image processing unit 152, the feature points in the image 1 and the image 2 are extracted in the feature point extracting unit 1521. As this feature point, for example, the highest part in the face such as the head of the nose is selected. The feature point is the same part in the image 1 and the image 2.
[0020]
After the feature points are extracted in this way, the three-dimensional shape recognition unit 1522 recognizes the three-dimensional shape of the user 14. Specifically, the image 1 and the image 2 are compared, and the relative position change amount between them is obtained. Since the relative position change amount is related to the distance between the user 14 and the cameras 12 and 13, the three-dimensional shape is recognized by performing a geometric calculation process based on the relative position change amount. Based on the three-dimensional shape, the calculation unit 1523 creates an image. In this way, the view morphing process appropriately reproduces a three-dimensional spatial image without using depth information.
[0021]
The image created as described above is sent to the display control unit 154 and stored in the image storage unit 153. The display control unit 154 performs pixel control according to the created image, and causes the display 11 to display the image.
In this manner, an image viewed from a viewpoint different from the viewpoints of the cameras 12 and 13 can be created and displayed. That is, as shown in FIGS. 1 and 4, a face image viewed from the front (FIG. 4A) and a face image viewed from the left (FIG. 4 (c)) can be created and displayed.
[0022]
In order to create an image that is as natural as possible, for example, a point that moves up, down, left, and right is displayed on the screen when the program starts, and the movement of that point is followed, and the user at that time is imaged with a camera. The image may be obtained in advance, and the image may be created using the image as appropriate. Thereby, a more natural image can be obtained.
[0023]
According to the present embodiment, images at various viewpoints can be created, images at various angles can be displayed, and even if the camera is not arranged in front of the object, It is possible to display an image of the front and function as a mirror.
[0024]
The image of the user 14 facing the front, which is created by the image processing unit 152, is displayed on the display 11, and when the user 14 processes and arranges the image, for example, the operation unit 11a of the display 11, 11b is used to instruct image processing. The image processing instruction may be performed using the operation units 11a and 11b of the display 11, or may be performed by key input using a keyboard, dragging using a mouse, or pen input using an electronic pen.
[0025]
This instruction is sent to the image processing unit 152, and the image processing unit 152 extracts an image captured from the image storage unit 153 and performs predetermined image processing on the image. More specifically, the image processing includes changing the face color and hair color of the image to an arbitrary color, adding imaginary items such as glasses, accessories, and costumes to the image, and storing the image in the image storage unit 153 in advance. The process of synthesizing this image with other images (such as people and backgrounds) stored in the image, such as the face parts (eyes, nose, mouth, ears, contours, etc.) and body parts (hands, Processing for deforming the legs, chest, belly, etc.).
[0026]
The image processed in this way is displayed on the display 11 under the control of the display control unit 154. The user 14 may further instruct image processing while viewing this image, or may transmit the image processing to another device as needed. When transmitting an image to another device, the communication control unit 155 transmits the image to another device via a line. For example, the image may be transmitted to a printer and printed out, or may be transmitted to another device (a server, a personal computer, or the like) via a network such as the Internet. This makes it possible to freely transmit a captured image or a processed image to another device.
[0027]
As described above, the electronic mirror system according to the present embodiment can perform image display in the same manner as an image projected on an actual mirror using images captured from different viewpoints, and furthermore, image processing, arrangement, Transmission can take place.
[0028]
With such a system, for example, it is also possible to transmit and receive an image to and from a cosmetics company to receive makeup guidance and consult skin troubles. In addition, it becomes easy to try on and match costumes on the Internet shopping. Further, by combining with a background image, a travel photograph or the like can be created. It is also possible to perform human divination using image analysis software.
[0029]
The present invention is not limited to the above embodiment, but can be implemented with various modifications. In the above embodiment, the case where the image is a still image is described. However, the present invention can be applied to a case where the image is a moving image.
[0030]
Further, in the above-described embodiment, a case is described in which a front-facing image is created using images captured from left and right viewpoints. In the present invention, a new virtual image is created from a plurality of images at different viewpoints. The present invention can be applied to general cases of creating an image. For example, by installing a camera behind the user and also using an image taken by the camera, an image of the rear view of the user can be created, thereby making it possible to check a hairstyle and the like. Become.
[0031]
【The invention's effect】
As described above, the electronic mirror system of the present invention can perform image display using images captured from different viewpoints in the same manner as an image projected on an actual mirror. Further, by providing the image processing means, it is possible to process, arrange, and transmit images.
[Brief description of the drawings]
FIG. 1 is a diagram showing a schematic configuration of an electron mirror system according to an embodiment of the present invention.
FIG. 2 is a block diagram showing an internal configuration of a control unit of the electronic mirror system shown in FIG.
FIG. 3 is a diagram for explaining a processing unit included in the image processing unit shown in FIG. 2;
FIG. 4 is a diagram showing an image captured and synthesized by the electron mirror system according to the embodiment of the present invention.
FIG. 5 is a diagram showing a schematic configuration of a conventional electron mirror system.
[Explanation of symbols]
11 Display 11a, 11b Operation Unit 12, 13 Camera 14 User 15 Control Unit 151 Image Acquisition Unit 152 Image Processing Unit 153 Image Storage Unit 154 Display Control Unit 155 Communication Control Unit 1521 Feature Point Extraction Unit 1522 3D Shape Recognition Unit 1523 Operation Unit

Claims (4)

  1. Display means arranged in front of the object to be imaged;
    At least two image pickup units installed at positions other than the arrangement position of the display unit, and imaging the object to be imaged from different viewpoints,
    An electronic mirror system comprising: a control unit that causes the display unit to display an image of the object to be captured from a viewpoint different from a viewpoint in the image from the images captured by the imaging unit.
  2. The control means obtains an image of the object facing forward using images of the object at different viewpoints imaged by two imagers arranged near both sides of the display means. The electronic mirror system according to claim 1, wherein
  3. The electron mirror system according to claim 1, wherein the control unit includes an image processing unit that performs image processing on a captured image and / or an obtained image.
  4. 4. The electron mirror system according to claim 1, wherein the control unit includes a communication control unit that transmits an image to another device.
JP2003091040A 2003-03-28 2003-03-28 Electronic mirror system Pending JP2004297734A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2003091040A JP2004297734A (en) 2003-03-28 2003-03-28 Electronic mirror system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2003091040A JP2004297734A (en) 2003-03-28 2003-03-28 Electronic mirror system

Publications (1)

Publication Number Publication Date
JP2004297734A true JP2004297734A (en) 2004-10-21

Family

ID=33404514

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2003091040A Pending JP2004297734A (en) 2003-03-28 2003-03-28 Electronic mirror system

Country Status (1)

Country Link
JP (1) JP2004297734A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006063407A1 (en) * 2004-12-17 2006-06-22 Leslie James Mcspadden An image capture and display arrangement
JP2009284361A (en) * 2008-05-23 2009-12-03 Non-Grid Inc Interactive mirror
JP2010050842A (en) * 2008-08-22 2010-03-04 Sony Taiwan Ltd High dynamic stitching method for multi-lens camera system
KR20110085033A (en) * 2010-01-19 2011-07-27 엘지전자 주식회사 Multi-display device and method of providing information using the same
WO2014140303A1 (en) * 2013-03-15 2014-09-18 Gosset Stéphane Panoramic photography system comprising a one-way mirror
WO2014156146A1 (en) * 2013-03-29 2014-10-02 パナソニック株式会社 Electronic mirror device
JP2016509683A (en) * 2012-12-18 2016-03-31 ヴィルコフスキー,ニッシ Apparatus, system and method for imaging and displaying appearance
US10469724B2 (en) 2014-05-16 2019-11-05 Panasonic Intellectual Property Management Co., Ltd. Imaging and displaying device and method of operation of imaging and displaying device

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006063407A1 (en) * 2004-12-17 2006-06-22 Leslie James Mcspadden An image capture and display arrangement
JP2009284361A (en) * 2008-05-23 2009-12-03 Non-Grid Inc Interactive mirror
JP4598842B2 (en) * 2008-05-23 2010-12-15 株式会社ノングリッド Interactive mirror
JP2010050842A (en) * 2008-08-22 2010-03-04 Sony Taiwan Ltd High dynamic stitching method for multi-lens camera system
KR20110085033A (en) * 2010-01-19 2011-07-27 엘지전자 주식회사 Multi-display device and method of providing information using the same
KR101666922B1 (en) * 2010-01-19 2016-10-17 엘지전자 주식회사 Multi-display Device And Method Of Providing Information Using The Same
JP2016509683A (en) * 2012-12-18 2016-03-31 ヴィルコフスキー,ニッシ Apparatus, system and method for imaging and displaying appearance
FR3003360A1 (en) * 2013-03-15 2014-09-19 Stephane Gosset Panoramic photographic system comprising a mirror without tain
WO2014140303A1 (en) * 2013-03-15 2014-09-18 Gosset Stéphane Panoramic photography system comprising a one-way mirror
WO2014156146A1 (en) * 2013-03-29 2014-10-02 パナソニック株式会社 Electronic mirror device
JP6008309B2 (en) * 2013-03-29 2016-10-19 パナソニックIpマネジメント株式会社 Electronic mirror device
US10198622B2 (en) 2013-03-29 2019-02-05 Panasonic Intellectual Property Management Co., Ltd. Electronic mirror device
US10469724B2 (en) 2014-05-16 2019-11-05 Panasonic Intellectual Property Management Co., Ltd. Imaging and displaying device and method of operation of imaging and displaying device

Similar Documents

Publication Publication Date Title
US9710973B2 (en) Low-latency fusing of virtual and real content
CN104871084B (en) Self adaptation projector
CN105283824B (en) With the virtual interacting of image projection
US8976160B2 (en) User interface and authentication for a virtual mirror
US8982110B2 (en) Method for image transformation, augmented reality, and teleperence
CN104699247B (en) A kind of virtual reality interactive system and method based on machine vision
US9779512B2 (en) Automatic generation of virtual materials from real-world materials
AU2014304760B2 (en) Devices, systems and methods of virtualizing a mirror
US20160128450A1 (en) Information processing apparatus, information processing method, and computer-readable storage medium
EP2988275A1 (en) Information processing device and information processing method, display device and display method, and information processing system
JP5936155B2 (en) 3D user interface device and 3D operation method
JP5966510B2 (en) Information processing system
CN102959616B (en) Interactive reality augmentation for natural interaction
US9495800B2 (en) Storage medium having stored thereon image processing program, image processing apparatus, image processing system, and image processing method
Waltemate et al. The impact of avatar personalization and immersion on virtual body ownership, presence, and emotional response
US20140176591A1 (en) Low-latency fusing of color image data
US10320437B2 (en) System and method for immersive and interactive multimedia generation
US20130335405A1 (en) Virtual object generation within a virtual environment
KR101183000B1 (en) A system and method for 3D space-dimension based image processing
AU2006282764B2 (en) Capturing and processing facial motion data
JP4869430B1 (en) Image processing program, image processing apparatus, image processing system, and image processing method
CA3003550A1 (en) Real-time visual feedback for user positioning with respect to a camera and a display
US20140225977A1 (en) Devices, systems and methods of virtualizing a mirror
US8055061B2 (en) Method and apparatus for generating three-dimensional model information
JP6027764B2 (en) Mirror system and control method thereof

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20051018

RD04 Notification of resignation of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7424

Effective date: 20060811

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20071206

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20071211

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20080207

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20080324