JP2006040083A - Mirror type information providing device, information providing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve mirror type information providing technology capable of indicating the same visual characteristics as a normal mirror in surface overview of the mirror and operating information presented in the mirror without touching on a surface of the mirror, and capable of cooperating information with the mobile telephone. <P>SOLUTION: The mirror type information providing technology composed of an image sensor 13 for imaging a front of a mirror 21 and a computer 14 to be connected to a network 20 in which the mirror 21 is configured by a half mirror 11 and a display 12. The computer 14 has optical signal detecting means 15 for detecting a position of an optical signal and an ID etc. of the optical signal to be transmitted by a mobile terminal of a user from an image inputted from the image sensor 13, and controlling means 16 for performing a process corresponding to the position of the optical signal and the ID etc. of the optical signal detected by the optical signal detecting means 15. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a technique in which a function for providing information on a network is added to a mirror.

  In research on ubiquitous computing, many systems have been proposed in which computers embedded in indoor walls and furniture support human activities in various forms. Several systems for presenting and manipulating information have been developed.

  For example, a system has been proposed in which information operation is performed by pointing image information virtually presented in a mirror with a hand of a self-image in the mirror (see, for example, Non-Patent Document 1). In such a system, in order to enable calculation of the positional relationship between the image information presented in the mirror and the self-image, a video mirror is used that displays a camera image taken on the front of the system on a display.

In addition, a mirror-type information presentation apparatus configured using a special optical system so that a plurality of images can be displayed in a mirror is proposed (see, for example, Non-Patent Document 2).
Eiichi Hosoya, Miki Kitabata, Hidenori Sato, Ikuo Harada, Hisao Nojima, Fumiharu Morisawa, Shinichiro Muto, “Mirror Interface for Real World Interaction”, Interaction 2003, pp. 95-96, 2003 Keuta Ushida, Kei Tanaka, Ken Naemura, Hiroshi Harashima, “Interaction / Information Environment Based on Mirror Metaphor i-mirror”, Journal of the Virtual Reality Society of Japan, Vol. 8, no. 4, pp. 421-431, 2003

  In order to realize a mirror-type information providing apparatus that can be used in the same way as an existing mirror, it is important to have a mirror function itself. All existing systems with a mirror motif use special optical systems that are different from video mirrors or ordinary mirrors. The mirror has a very specific visual effect. In building interior coordination, mirrors are often placed in consideration of this visual effect. Therefore, unless the visual effect of the existing mirror is completely provided, it cannot be a mirror-type information providing apparatus that can be used instead.

  For the user, the original mirror is only for viewing. On the other hand, the operation of pointing the information content presented in the mirror and acquiring it in the information terminal is an unprecedented interaction with the mirror. In implementing a new interaction with a mirror having such characteristics, it is necessary to design an appropriate user interface with due consideration for the original role of the mirror.

  In order to point the display in front of the user, the touch panel method is excellent in operability. However, it is not desirable for the mirror to have scratches or dirt on its surface for its original use. For this reason, an operation method that directly touches the mirror surface is not appropriate.

  In view of such circumstances, the present invention has been made. The purpose of the present invention is as follows: (1) The surface appearance of the mirror exhibits visual characteristics equivalent to those of a normal mirror, and (2) the information presented in the mirror is mirrored. It is possible to operate without touching (3) to realize a mirror-type information providing technology that enables information linkage with a mobile phone.

  In the mirror-type information providing apparatus of the present invention, a half mirror is used to realize the same surface appearance as that of an existing mirror. Information presentation in the mirror is performed by a display in close contact with the back of the half mirror. When the display is blacked out, the surface appearance is equivalent to a normal mirror. In addition, an image sensor, an optical signal detection means, and a computer connected to a network are provided for manipulating the information in the mirror by the optical signal. The optical signal detection means detects the position of the optical signal and its ID from the video input by the image sensor, and the computer performs processing corresponding to the optical signal detection position and the ID.

  In order for the user to point to the presentation information in the mirror using an optical signal, some visual feedback is required. For this purpose, the computer performs a process of presenting a cursor at a position in the display corresponding to the position where the optical signal is detected.

  In addition, when the user clicks on the presentation information in the mirror with the optical signal, the optical signal is the source of the optical signal in order to transmit the related information to the portable terminal that is the source of the optical signal. The mobile terminal ID for specifying the mobile terminal is transmitted and used.

  That is, the computer displays a plurality of thumbnail images corresponding to the information content on the network on the display, and an optical signal that transmits the portable terminal ID is detected at a position corresponding to any of the thumbnail images. In this case, information (such as URL of information content) corresponding to the thumbnail image is transmitted to the mobile terminal specified by the mobile terminal ID.

  Further, in order to allow the user to paste arbitrary image information in the mirror, the ID transmitted by the optical signal is also used as a content ID for specifying predetermined information content.

  In this case, when an optical signal conveying the content ID is detected, the computer displays a thumbnail image of the information content specified by the content ID at a position on the display corresponding to the position where the optical signal is detected. .

  Further, the optical signal detecting means provided in the mirror type information providing apparatus of the present invention is mounted so as to detect a color blinking signal using a plurality of colors as the optical signal. For example, it is mounted so that a hue difference optical signal that can be transmitted using a color display provided in an existing portable terminal as a light source can be detected.

  Thereby, the existing portable terminal is linked to the mirror-type information providing apparatus of the present invention without adding any special device to them.

  The mirror-type information providing apparatus according to the present invention has a configuration in which a half mirror and a display are attached, so that the surface appearance is the same as a normal mirror. By using a thin and light liquid crystal display, the volume can be reduced and it can be installed on an indoor wall.

  In addition, since the optical signal is used for the operation of the presentation information in the mirror, the operation can be performed without directly touching the mirror surface. That is, the operation can be performed without polluting the mirror surface.

  In addition, when an optical signal is used, it is only necessary to provide a small image sensor for optical signal detection on the mirror side, which is advantageous in terms of apparatus cost and miniaturization as compared with a case where a touch panel device is provided.

  In addition, an optical signal using a hue difference that can be transmitted using a color display of a mobile phone can be used as the optical signal.

  In addition, an optical signal using an infrared light source provided in some mobile phones can be used. Therefore, they can be linked to the mirror-type information providing apparatus of the present invention without any change to the existing mobile phone.

(Embodiment 1)
A basic configuration example of the mirror-type information providing apparatus of the present invention is shown in FIG.

  As shown in FIG. 1, the mirror-type information providing apparatus includes a mirror 21 including a half mirror 11 and a display 12, an image sensor 13, and a computer 14 connected to a network 20. The computer 14 detects an optical signal. It comprises means 15, control means 16, memory 17, image output means 18, and network interface 19.

  The display unit of the display 12 is in close contact with the back surface of the half mirror 11. Therefore, the surface appearance of the half mirror 11 is not different from a normal mirror when the display 12 is blacked out.

  The image sensor 13 is installed on a mirror 21 including the half mirror 11 and the display 12 so as to capture the front of the mirror 21.

  The computer 14 has a series of hardware and software included in a general-purpose personal computer. Furthermore, the image sensor 13 is characterized in that it includes an optical signal detection means for processing an image picked up and detecting an optical signal.

  Further, the optical signal detection means 15 may be implemented as hardware such as a DSP (Digital Signal Processor). Note that the DSP is a microprocessor specialized for processing of voice, images, and the like, and is hardware that is installed in a modem, a personal computer, etc. and performs predetermined processing at high speed. The optical signal detection means 15 can also be implemented as software that operates on the computer OS.

  The control means 16 included in the computer 14 displays thumbnail images corresponding to various information contents acquired via the network 20 on the display 12 by the image output means 18. As a result, various information is presented in the mirror 21. Further, the computer 14 performs processing on various presented information based on the optical signal detected by the optical signal detection means 15.

  FIG. 2 shows an overview of the mirror-type information providing apparatus actually developed. As shown in FIG. 2, the display 11 and the computer 14 can be embedded in a wall.

  Hereinafter, the use of the mirror-type information providing apparatus will be described with reference to the drawings.

(Show cursor)
First, the computer 14 presents a cursor indicating which position the user is pointing in the mirror 21 so that the user can point an arbitrary thumbnail image presented in the mirror 21 by an optical signal. How the computer 14 processes the cursor presentation will be described with reference to FIG.

  As shown in FIG. 3, a user 31 holds a light source 32 of an optical signal of a mobile terminal such as a mobile phone, and transmits an optical signal toward a mirror 21 including a half mirror 11 and a display 12. As this optical signal, a simple luminance difference blinking signal or a color blinking signal using a hue difference can be used.

Specifically, optical signals described in the following technologies 1 to 4 can be used.
(Technology 1) Nobuyuki Matsushita, Daisuke Hihara, Teruyuki Gohara, Shinichi Yoshimura, Junichi Kyokumoto, “ID Cam: Smart camera capable of acquiring scene and ID simultaneously”, Transactions of Information Processing Society of Japan, vol. 43, no. 12, pp. 3664-3674, Dec. 2002
(Technology 2) D. J. et al. Moore, R.A. Want, et al. “Implementing Phonics: Combining Computer Vision with Infrared Technology for Interactive Physical Icon”, Proceedings of ACM UIST '99, Ashville, N .; C. , November 8th-10th, pp. 67-68, 1999
(Technology 3) Tsuyoshi Aoki, “Infrared tags read by cameras and their applications” Interactive System and Software VIII, pp. 131-136, Modern Science, 2000
(Technology 4) Kento Miyaku, Go Higashino, Yoshinobu Tonomura, “C-Blink: Hue-difference optical signal marker by mobile phone display”, Interaction 2004, pp. 153-160, 2004.

  In particular, when the hue difference optical signal method described in the technique 4 is used, a color display of a portable terminal can be used as a light source.

  First, the input image of the image sensor 13 is sent to the optical signal processing means 15. Then, after the left / right reversal processing is performed, the continuous video frames are processed to detect the position and ID of the optical signal.

  Specific processing contents differ depending on the optical signal to be used. The processing described in the techniques 1 to 4 is performed by the optical signal, and the position and the ID are detected.

On the other hand, an optical signal that presents a predetermined shape marker can be used instead of a continuous blinking signal. For example,
(Technology 5) Junichi Kyokumoto, “Augmented Reality Construction Method Using Two-dimensional Matrix Code”, Interactive and Software IV, pp. 199-208, Modern Science, 1996
Can be used as an optical signal. In addition, it is possible to use a geometric pattern having a characteristic shape that can uniquely identify an ID and detect its position in an image, a character cartoon, or the like as an optical signal.

  After the optical signal area and the center of gravity or the center coordinates are detected by the optical signal detection means 15 by the above procedure, the control means 16 performs a process of presenting the cursor 33 at the coordinates on the display 12 corresponding to the center of gravity or the center coordinates. Do. For example, if the size of the input video is 160 × 120 and the resolution of the display 12 is 1600 × 1200, the coordinates on the display 12 corresponding to the coordinates (X, Y) in the video are (X × 10, Y × 10).

  The computer 14 performs the above procedure in real time. As a result, when the user moves the optical signal source at hand, the presented cursor moves in accordance with the movement. In addition, the optical signal used for cursor presentation shall transmit ID which shows that it is a signal for cursors, or shall not contain ID information.

(Acquisition of information to mobile terminals)
Next, a function of acquiring information related to the thumbnail image presented in the mirror 21 to the portable terminal 32 serving as the light source of the optical signal will be described. In the mirror-type information providing apparatus, new information on the network 20 is automatically presented as needed by information providing application software operating in the computer 14. The information providing application software may be a so-called Web browser and its contents. Also,
(Technology 6) Keita Watanabe, Michiyasu Yasumura, “Memorium: Proposal and Prototype of Viewing Interface”, WISS 2002 Papers, pp. 99-104, 2002
It may be an application that uses an information display method that is intended to make the user aware of the presence of information as proposed in.

  At this time, a plurality of thumbnail images 41 are presented on the display 14 as shown in FIG. In addition, the cursor 33 is presented at the position where the optical signal for cursor presentation is detected. As a result, information is presented in the mirror 21 as shown in FIG.

  Here, a process of clicking one of the thumbnail images 41 presented in the mirror 21 and acquiring the related information (URL) in the portable terminal 32 as a light source will be described with reference to FIG.

  The user 31 moves the cursor 33 presented in the mirror 21 onto the desired thumbnail image 41 by moving the portable terminal 32 at hand. Then, by pressing a predetermined button on the mobile terminal 32, an optical signal for transmitting an ID (mobile terminal ID) for specifying the mobile terminal 32 is transmitted.

  The optical signal detection means 15 detects the optical signal for transmitting the mobile terminal ID from the image sensor video in the same procedure as the detection of the optical signal for cursor presentation, and the position coordinates (the center of gravity or the center of the area) and The mobile terminal ID is notified to the control means 16.

  The control means 16 processes this as having occurred by the mobile terminal 32 specified by the mobile terminal ID at the coordinates on the display 12 corresponding to the notified position coordinates.

  That is, the information providing application software controlled by the control unit 16 displays a plurality of thumbnail images 41 on the display 12, and the portable terminal 32 identified by the portable terminal ID on one of the thumbnail images 41. Handles the event that a click occurred.

  Here, as processing contents associated with this event, more various processes can be associated by mounting the information providing application.

  In order to realize the function of causing the portable terminal 32 to acquire the clicked information, the control means 16 performs the following processing.

  First, the control means 16 or the information providing application software operating on it manages the mobile terminal addresses corresponding to the respective mobile terminal IDs as shown in FIG. When the portable terminal ID is detected by the above procedure, first, the thumbnail image 41 displayed at the coordinates on the display 12 corresponding to the detected position coordinates is detected. The control means 16 or the information providing application software similarly manages the URL corresponding to each thumbnail image, and the URL corresponding to the thumbnail image clicked by the optical signal carrying the portable terminal ID corresponds to the portable terminal ID. To the address via the network 20.

  In FIG. 7, each mobile terminal address is managed as an e-mail address. Accordingly, in this example, the URL information is transmitted as e-mail. Each mobile terminal address can be managed as an IP address and transmitted as an IP packet.

  In addition, an address and a transmission method according to the protocol of the network 20 to be used can be used.

  Also, the mobile terminal address information corresponding to each mobile terminal ID is not held by the computer 14 itself, but is held by a predetermined server on the network, and is implemented so that the computer 14 accesses and uses it as necessary. May be.

(Paste information from the mobile device into the mirror)
A function of pasting image information in the mirror 21 by the portable terminal 32 will be described with reference to FIG. As shown in FIG. 8, the user 31 moves the cursor 33 presented in the mirror 21 with the portable terminal 32 at hand, and moves the cursor 33 to a position where image information is to be pasted. Then, by pressing a predetermined button on the portable terminal 32, an optical signal for transmitting a content ID specifying image information is transmitted.

  The optical signal detection means 15 detects the optical signal for transmitting the content ID from the image of the image sensor 13 in the same procedure as the detection of the optical signal for cursor presentation, and the position coordinates (the center of gravity or the center of the area) and The control unit 16 is notified of the content ID.

  The control means 16 detects that an operation of pasting image information specified by the content ID at the coordinates on the display 12 corresponding to the position coordinates has occurred, and processes this.

  Specifically, as shown in FIG. 9, the correspondence between the content ID and the thumbnail image 41 is held, and the image corresponding to the content ID is displayed on the display 12 at the position corresponding to the detected position coordinate of the optical signal of the content ID. To present.

  The above-described acquisition function of the presentation information to the portable terminal 32 and the function of pasting information from the portable terminal 32 into the mirror 21 basically use the optical signal to the display 12 described in (Technology 4). It can be implemented with the same mechanism as the interaction function used.

  In the above example, the acquisition and pasting of information has been described. However, it is also possible to implement a process in which normal Web content is displayed in the mirror 21 and the anchor is clicked by an optical signal. For example, it can be implemented by processing that a so-called click has occurred in the position coordinates on the display 12 corresponding to the position coordinates where the optical signal for transmitting the portable terminal ID is detected.

  In addition to these processes, the optical signal detection means 15, the control means 16, and the application on the control means 16 in the computer 14 may be implemented so that operations such as dragging and double clicking using an optical signal can be performed. .

  Since the optical signal detection means 15 basically functions as a pointing device using an optical signal, the mounting thereof is the same as the processing in a normal pointing device. However, any operation can be performed without touching the mirror surface by using the optical signal.

(Embodiment 2)
FIG. 10 is a configuration diagram of a mirror-type information providing apparatus showing Embodiment 2 of the present invention.

  The mirror-type information providing apparatus is assumed to be used as a terminal of a so-called home network system. Therefore, it is used as a part of a home network system as shown in FIG.

  The mirror-type information providing apparatus type 1 shown in FIG. 10 has the same configuration as that described with reference to FIG. 1, and is connected to the home server 101 and the Internet via a wired LAN.

  On the other hand, the mirror-type information providing device type 2 and the mirror-type information providing device type 3 show other configurations intended to improve the portability and size of the device.

  In other words, the mirror-type information providing apparatus type 2 includes a wireless communication unit 102 in addition to the configuration shown in FIG. 1, and is configured to be connected to a network wirelessly. Many existing mirrors are portable and can be moved and installed anywhere in the house. This is a configuration for realizing portability similar to that of such a normal mirror.

  Further, in the mirror type information providing apparatus type 3, the computer portion is separated and provided in the home server 101 in order to further reduce the size. In this configuration example, the computer 14 portion in FIG. 1 is incorporated in the home server 101. The mirror part is composed of a half mirror 11, a display 12, an image sensor 13, and wireless communication means 102. The input video of the image sensor 13 and the image information output to the display 12 are communicated with the home server 101. Sent and received. The means and processing contents of the computer in this configuration are the same as described above except that the input destination is changed to the means for communicating with the mirror part.

  Note that some or all of the processing functions of the apparatus shown in the present invention can be configured as a program and realized using a computer. In addition, a computer-readable recording medium such as a flexible disk, MO, ROM, or memory card can be used to store a program for realizing the processing function of each unit by the computer or a program for causing the computer to execute the processing procedure. It can be recorded on a CD, a DVD, a removable disk, etc., stored, provided, and distributed via a communication network such as the Internet.

The basic block diagram of a mirror-type information provision apparatus. Schematic view of a mirror-type information providing device. Explanatory drawing of the cursor presentation process in a mirror type information provision apparatus. The figure which shows the example of a display display of a mirror-type information provision apparatus. The figure which shows the in-mirror information presentation example of a mirror-type information provision apparatus. Explanatory drawing of the information processing in a mirror type information provision apparatus. The figure which shows the example of the conversion table of portable terminal ID and a transmission destination address. Explanatory drawing of the information sticking in a mirror type information provision apparatus. The figure which shows the example of the conversion table of content ID and related information. The figure which shows the structural example of the mirror-type information provision apparatus connected to a home network.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Half mirror 12 ... Display 13 ... Image sensor 14 ... Computer 15 ... Optical signal detection means 16 ... Control means 17 ... Memory 18 ... Image output means 19 ... Network interface 20 ... Network, such as the internet and LAN 21 ... Mirror 31 ... User 32 ... mobile terminal 33 ... cursor 41 ... thumbnail image 101 ... home server 102 ... wireless communication means

Claims (15)

A mirror that displays information consisting of a half mirror and a display installed on the back of the half mirror;
An image sensor for imaging the front of the mirror;
A mirror-type information providing device comprising a computer connected to a network,
The computer
Optical signal detection means for detecting at least the position of the optical signal transmitted from the mobile terminal of the user, the ID of the optical signal, and optical signal pointing from the video input from the image sensor;
Control means for performing processing corresponding to the position of the optical signal detected by the optical signal detection means, the ID of the optical signal, and optical signal pointing;
A mirror-type information providing apparatus comprising:   The mirror-type information providing apparatus according to claim 1, wherein the control unit performs a process of presenting a cursor at a position in the display corresponding to a position where the optical signal is detected.   3. The mirror-type information providing apparatus according to claim 1, wherein an ID of the optical signal is used as a mobile terminal ID for specifying a mobile terminal that transmits the optical signal. The computer displays a thumbnail image corresponding to information content on the network in a display,
When the pointing position by the optical signal corresponds to the position of the thumbnail image, the control means transmits information corresponding to the thumbnail image to the portable terminal specified by the optical signal ID. Item 4. The mirror-type information providing apparatus according to any one of Items 1 to 3.   The mirror-type information providing apparatus according to any one of claims 1 to 4, wherein an ID transmitted by the optical signal is used as a content ID for specifying predetermined information content.   When the control means detects pointing by the optical signal specifying the content ID, the thumbnail of the information content specified by the content ID at a position on the display corresponding to the position where the optical signal is detected. 6. The mirror type information providing apparatus according to claim 5, wherein an image is displayed.   7. The mirror type information providing apparatus according to claim 1, wherein the optical signal detected by the optical signal detecting means is a color blinking optical signal configured using a plurality of colors. A mirror that displays information consisting of a half mirror and a display installed on the back of the half mirror;
An image sensor for imaging the front of the mirror;
An information providing method in a mirror-type information providing apparatus comprising a computer connected to a network,
The computer
The optical signal detection means detects at least the position of the optical signal transmitted by the user's mobile terminal, the ID of the optical signal, and the optical signal pointing from the video input from the image sensor,
The control means performs processing corresponding to the position of the optical signal detected by the optical signal detection means, the ID of the optical signal, and optical signal pointing.
An information providing method characterized by comprising:   The information providing method according to claim 8, wherein the control unit performs a process of presenting a cursor at a position in the display corresponding to a position where the optical signal is detected.   The information providing method according to claim 8 or 9, wherein the ID of the optical signal is used as a mobile terminal ID for identifying a mobile terminal that transmits the optical signal. The computer displays a thumbnail image corresponding to information content on the network in a display,
When the pointing position by the optical signal corresponds to the position of the thumbnail image, the control means transmits information corresponding to the thumbnail image to the portable terminal specified by the optical signal ID. Item 11. A method for providing information according to any one of Items 8 to 10.   The information providing method according to any one of claims 8 to 11, wherein an ID transmitted by the optical signal is used as a content ID for specifying predetermined information content.   When the control means detects pointing by the optical signal specifying the content ID, the thumbnail of the information content specified by the content ID at a position on the display corresponding to the position where the optical signal is detected. 13. The information providing method according to claim 12, wherein an image is displayed.   The information providing method according to claim 8, wherein the optical signal detected by the optical signal detecting means is a color blinking optical signal configured using a plurality of colors.   A program characterized in that the mirror-type information providing apparatus or the information providing method according to any one of claims 1 to 14 is described as a computer program and can be executed.