JP2011186007A - Display control device and information display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To surely detect both of a user's intentional action and unintentional action, without complicated processing, to make seamless control. <P>SOLUTION: An information display device 20 includes: an image pickup part 22 which picks up an image in a real space; a display part 26 which displays the picked-up image; a detection part 21 which detects a state between the eyebrows of a person who visually recognizes the display part 26; and a positional information-obtaining part 23 which obtains the information on the current position of the real space. On the basis of the positional information thus obtained, an additional information-processing part 24 conducts at least either processing for obtaining additional information which is correlated with the positional information, or processing for generating additional information in association with the positional information, and controls a state in which the processed additional information is superposed on the picked-up image and is then displayed, on the basis of the state between the eyebrows which is detected by the detection part 21. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a display control device and an information display device that control the display state of information and additional information, and in particular, a display that enables a person viewing information to control the display state of additional information consciously or unconsciously. The present invention relates to a control device and an information display device. Here, the information includes information that is normally displayed on an information display device and the like, and information such as an object in front of the human eye that is visually recognized, and the additional information is defined in the present invention using the means described in the present invention. It is information that can be additionally displayed or changed.

  In recent years, technologies for displaying augmented reality (AR) that superimposes and displays additional information in an actual environment have been proposed in various forms. In augmented reality, information of the virtual world is superimposed and displayed as additional information on the visual information of the real world. Therefore, the amount of information in the world where such visual information is mixed inevitably increases. If the amount of additional information increases too much, it will hinder recognition of visual information in the real world. Some device for controlling the amount of additional information is required.

  As an input device that controls a general external device, for example, it includes at least one sensor that detects a change in the epidermis associated with the movement of the human lower jaw, and a signal generation unit that generates a control signal based on the change in the epidermis. As a variation of the epidermis, it is known that the variation of the epidermis of the temple is detected in a non-contact manner, thereby generating a control signal and outputting it to an external device (Patent Document 1).

JP 2008-304451 A

  However, the input device disclosed in Patent Document 1 described above is intended only for control based on conscious movements by the user in order to detect fluctuations in the epidermis of the temple, and seamless including more natural unconscious movements. There is a problem that it is impossible to perform proper control.

  In order to eliminate the above-described problems caused by the conventional technology, the present invention can perform seamless control by reliably detecting both conscious and unconscious operations by a user without requiring complicated processing. It is an object to provide a display control device and an information display device that can be used.

  A display control device according to an aspect of the present invention includes a detection unit that detects a state between eyebrows of a human who is viewing information, and adds the information and the information according to the state between the eyebrows detected by the detection unit. Control means for controlling the display state of at least one of the additional information to be displayed.

  The detection means is, for example, an optical sensor attached to a glasses-type device worn by the person. The glasses-type device is, for example, a head mounted display (HMD).

  An information display device according to an aspect of the present invention includes a display unit that displays information, a detection unit that detects a state between eyebrows of a human viewing the information, and a state between the eyebrows detected by the detection unit. And a control means for controlling a display state of at least one of the information and additional information added to the information.

  The control means may control, for example, display of additional information added to the information, or may control enlargement / reduction of a display state of at least one of the information and the additional information, or the information In addition, the playback speed of the display state of at least one of the additional information may be controlled.

  An information display device according to another aspect of the present invention is an information display device that displays augmented reality that is output in a state where additional information added to the image can be recognized on an image in the real space, Imaging means for capturing an image of the real space; display means for displaying the captured image; detection means for detecting a state between human eyebrows; and position information acquisition means for acquiring positional information of the real space; And performing at least one of a process of acquiring additional information associated with the position information and a process of generating additional information associated with the position information based on the acquired position information, and detected by the detection means And additional information processing means for controlling a superimposed display state of the processed additional information on the captured image based on the state between the eyebrows.

  According to the present invention, seamless control can be performed by reliably detecting both conscious and unconscious actions by a user without requiring complicated processing.

1 is a block diagram illustrating an overall configuration of a display control device according to a first embodiment of the present invention. It is a figure which shows the example of mounting | wearing of the detection part in the display control apparatus. It is a figure which shows the relationship between the distance from the human eyebrow center part, and the width | variety of an output value. It is a block diagram which shows the whole structure of the information display apparatus which concerns on the 2nd Embodiment of this invention. It is a front view which shows the example of an external appearance of the same information display apparatus. It is a figure for demonstrating the operation example of the information display apparatus. It is a figure for demonstrating the operation example of the superimposed display of the same information display apparatus. It is a figure for demonstrating the operation example of the superimposed display of the same information display apparatus. It is an external view of the information display apparatus which concerns on the 3rd Embodiment of this invention. It is a flowchart which shows the calibration process which concerns on the 4th Embodiment of this invention.

  Embodiments of a display control device and an information display device according to the present invention will be described below in detail with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is a diagram showing the overall configuration of the display control apparatus 10 according to the first embodiment of the present invention.
As shown in FIG. 1, the display control device 10 outputs a control signal for controlling the display state of the display device 18 to the display device 18, and detects the state between the eyebrows of a human viewing the display device 18. And a control unit 15 that generates a control signal for controlling the display state of the display device 18 based on the detection result from the detection unit 11.

  The detection unit 11 includes, for example, an optical sensor (such as a photoreflector), a light irradiation unit 12 such as an infrared LED that irradiates light on the human skin, and a reflected light reflected by the skin to detect a light reception signal. And a light-receiving unit 13 such as a phototransistor that outputs as follows.

  The control unit 15 converts the analog detection signal from the detection unit 11 into a digital signal, and the display state of the display device 18 based on the digital signal converted by the A / D converter 16. And a processing unit 17 that generates a control signal for controlling the control signal.

  In the display control apparatus 10 configured as described above, the detection unit 11 includes, for example, a human eyebrow M via an attachment jig 11a that can be attached to the glasses Q placed on the human face K as shown in FIG. Is attached so as to be able to detect the state of the human eyebrow M (for example, any one of the state of the epidermis based on the movement between the eyebrows and its variation).

  Specifically, as shown in FIG. 3, the detection unit 11 detects any one of the state and fluctuation of the epidermis between the eyebrows M connecting the human eyebrow MGs in a straight line, without contact, It is desirable to arrange via the mounting jig 11a. According to the experiment by the present applicant, the location where the detection unit 11 is arranged includes a position where L = 14 mm to 18 mm away from the midpoint P of the line segment S connecting the eyebrow MGs linearly to each eyebrow side. It has been found that a position where the region R can be detected is most preferable. Therefore, it is preferable that the detection unit 11 detects the region R in a non-contact manner.

  This is considered to be largely related to the characteristics of the movement of the epidermis in the human eyebrow M. That is, when the operation of bringing the eyebrows M is performed, the epidermis at the center of the eyebrows bulges and wrinkles around the eyebrows MG. At this time, the movement of the epidermis beside the eyebrow MG is more severe than the center of the raised eyebrows, so that the light emitted from the light irradiation unit 12 detected by the light receiving unit 13 is reflected by the epidermis. The change in the amount of reflected light also increases. Thereby, it can be said that the location where the width of the output value is the maximum is a location away from the central portion between the eyebrows by a distance L.

  In order to suitably detect the movement between the eyebrows, the photodetector (ROPR RPR-220) is used as the light detection unit 11 to irradiate the epidermis with infrared light, and the amount of reflected light due to changes in the skin surface. The change may be detected by an infrared sensor in a non-contact manner. In this case, it is efficient to measure the output of the photo reflector as a numerical value in 256 steps using Gainer (http://gainer.cc/), and to perform information addition processing using Jitter (registered trademark) as a programming language. is there.

  The output value width is calculated as follows. First, in a state where the display control device according to the present invention is mounted, a set of a series of movements that do not close the eyebrows, close to the limit, or close, do a total of 3 sets measurement, and from the obtained 3 sets of data, Find the maximum and minimum values for each set. Next, from the three maximum values and the three minimum values obtained, the smallest maximum value and the largest minimum value are set as the maximum value and the minimum value of the output value, respectively, and the width of the output value is obtained from the difference.

  Further, although there are individual differences depending on the subject, it has been found that the output values of 26 levels at the minimum and 89 levels at the maximum can be detected at the position where the width of the output value of the detection unit 11 is maximum. Therefore, if the control signal is generated by the control unit 15 according to the stage of the output value, the operation of the display device 18 can be controlled seamlessly over a plurality of stages.

The display control apparatus 10 according to the present embodiment has the following effects.
That is, the movement of the eyebrow M can be performed consciously moving the eyebrow M when the person tries to stare at something. It is recognized that For this reason, if this display control apparatus 10 is used, it is possible to reliably detect both conscious and unconscious movements of the person viewing the display apparatus 18 and control the display state of the display apparatus 18. It becomes possible. Display control includes, for example, enlargement / reduction of the image displayed on the display device 18, change of the video playback speed, display control of content in the layer structure (such as seeing the answer to the problem when the eyebrows are close), and display information on the television It is conceivable to switch the amount or content, or switch the channel.

[Second Embodiment]
FIG. 4 is a block diagram showing the configuration of the information display device 20 according to the second embodiment of the present invention, and FIG. 5 is a front view showing an example of the appearance of the device 20.
The information display device 20 according to the second embodiment controls the amount of augmented reality (AR) information.

  The information display device 20 includes a detection unit 21 having the same function as the detection unit 11 according to the first embodiment, and an imaging unit 22 that captures an image of a real space. The information display device 20 also includes a position information acquisition unit 23 that acquires position information indicating the position and direction in the real space, and an additional information that performs processing of additional information based on the position information acquired by the position information acquisition unit 23. And an information processing unit 24.

  Further, the information display device 20 displays an image of the real space imaged by the imaging unit 22 and superimposes the additional information generated by the additional information processing unit 24 on the image of the real space to give an augmented reality. A display unit 26 is provided.

  In this embodiment, as shown in FIG. 5, the present invention is applied to a portable terminal type information display device 20. Unlike the previous embodiment, the detection unit 21 includes an imaging unit such as a CCD camera provided in the upper center portion of the front surface of the information display device 20 (on the display screen 26a side), and an internal image recognition processing unit (not shown). It is comprised by. The image recognition processing unit recognizes the image information of the imaging unit to grasp the image information of a predetermined area between the eyebrows, detects the state of the image information, and outputs the detected value.

  The imaging unit 22 is composed of a camera device such as a CCD or C-MOS, for example, and is provided on the side opposite to the display screen 26a of the information display device 20 as shown in FIG. The user captures an image of the real space in the direction in which the imaging unit 22 is directed.

  The position information acquisition unit 23 includes a device such as a GPS module or an electronic compass, for example, and acquires the current position and orientation in the real space as position information. The additional information processing unit 24 acquires the additional information associated with the acquired position information from, for example, the additional information providing server 29 connected via the Internet 28, or associates the additional information with the acquired position information. Process to generate. Further, the additional information processing unit 24 controls the output mode of the processed additional information on the display unit 26 based on the detection result of the detection unit 21. The display unit 26 is composed of a display device or the like, and displays an augmented reality composed of a captured image and processed additional information on the display screen 26a.

  Note that the additional information includes at least one of character information, image information, and audio information added in association with a visually recognizable object existing on the captured image. When the additional information is at least one of character information and image information, the additional information processing unit 24 detects at least one of the transparency and the display amount in the superimposed display on the augmented reality of the additional information. Control is performed step by step according to the control signal from 21.

The information display device 20 configured as described above operates as follows, for example.
The user directs the imaging unit 22 toward an object for which additional information is to be viewed, such as a store or a company. As shown in FIG. 6A, the detection unit 21 detects this when the user is in a normal state where the eyebrow gap M is not close. Therefore, as shown in FIG. Only the real space image captured by the imaging unit 22 is displayed on the display screen 26a.

  In this state, as shown in FIG. 6C, when the user unconsciously or consciously wrinkles between the eyebrows by staring at the display screen 26a, the detection unit 21 detects this, As illustrated in FIG. 6D, the information processing unit 24 adds, to the real space image displayed on the display screen 26 a, additional information 31, 32 related to the visible object 30 existing in the image. 33 and 34 are displayed in a superimposed manner. In this example, the additional information 31 and 34 is character information, and the additional information 32 is image information. Further, the additional information 33 is audio information, and as an output mode in this case, it is considered that audio is output in addition to the display output.

  The display amount of the additional information 31 to 34 can be controlled step by step according to the degree of wrinkles between the eyebrows. Conversely, in the normal state, the maximum additional information is displayed, and when the display screen 26a is stared and wrinkles occur between the eyebrows, the additional information can be controlled to be gradually deleted according to the degree of wrinkles. . Generally, when staring at the display screen 26a, a part of the image in the real space may be watched, and in such a case, the display of the additional information is disturbed.

  Note that the additional information processing unit 24 may control the transparency of the display of the additional information, for example, according to the stage of the output value detected by the detection unit 21. As a specific example, the additional information 31 that is character information will be described as an example. As shown in FIG. 7A, when the eyebrow gap M is not so close, the additional information 31a with high transparency is displayed in a superimposed manner, and as shown in FIG. 7B, the eyebrow gap M is close. In this case, the additional information 31b having low transparency is displayed in a superimposed manner.

  Also, as shown in FIG. 8A, when the detected output value is small, the amount of additional information 31c to be superimposed is small, and as shown in FIG. When it is large, it can be controlled so that the amount of the additional information 31d to be superimposed is increased.

  It has been confirmed that the human eyebrows M move not only when gazing consciously but also when staring unconsciously. Therefore, if the augmented reality is displayed in this way, not only when the object 30 is consciously stared but also when the object 30 is unconsciously added, it is added step by step according to the output value. The information 31 to 34 can be displayed in detail or can be displayed easily, and display / non-display of additional information or a change in display state can be realized by a seamless so-called fader operation.

  Note that in the case of image information such as the additional information 32, for example, the region of the superimposed image is controlled so as to change according to the output value of the detection unit 21, or audio information such as the additional information 33 is used. In some cases, for example, the sound output volume may be controlled so as to change according to the output value of the detection unit 21. In addition, the additional information may be enlarged or reduced, the color may be changed, or the content may be changed. Various output modes such as display switching can be controlled.

[Third Embodiment]
FIG. 9 is an external view showing an information display device according to the third embodiment of the present invention.
The information display device 20 according to the second embodiment described above is a portable terminal type, but the information display device according to the third embodiment is a head-mounted display (HMD) type display device. The block configuration is the same as the configuration shown in FIG.

  In the present embodiment, the information display device HMD is a glasses type, and a portion facing the eyes is a display screen 26a. The imaging unit 22 is attached to the upper center of the information display device HMD (of the glasses-type portion), and the detection unit 21 is disposed on the side thereof. The detection unit 21 may have any configuration of the first and second embodiments.

  The information display device HMD configured as described above can be applied to display of additional information (such as a surgical site and precautions) in a medical treatment (for example, surgery) system. Specifically, when applied to the treatment system, the doctor wears the information display device HMD, and the doctor confirms the augmented reality displayed on the display screen 26a while imaging the treatment location with the imaging unit 22. It is conceivable to perform treatment. In this case, for example, additional information necessary for the treatment can be displayed as a seamless augmented reality by both conscious operation and unconscious operation without requiring complicated processing.

[Fourth Embodiment]
FIG. 10 is a flowchart showing the calibration process of the detection units 11 and 21.
Since the state between the eyebrows varies from person to person, it is desirable to perform calibration processing in advance. This processing is performed by the processing unit 17 in FIG. 1 or the additional information processing unit 24 in FIG.
First, the outputs of the detectors 11 and 21 are sampled for a certain period of time with the eyebrows close to the limit (S100). Next, in the normal state between the eyebrows, the outputs of the detection units 11 and 21 are similarly sampled for a predetermined time (S102). The smallest difference between the detected output values sampled between the eyebrows and the normal state is set as the detected value width in the processing unit 17 and the additional information processing unit 24 (S104). Then, one or a plurality of threshold values are calculated and set from the obtained width of the detected value (S106). As a result, accurate detection is possible without individual differences.

  In addition to the applications described above, the present invention can be applied to a three-dimensional television using liquid crystal shutter glasses. In this case, for example, the amount of display information transmitted as layer information, such as a telop or a clock, can be controlled and the content can be switched by the detection unit. In 3D television, since synchronous communication is performed between the display device and the liquid crystal shutter glasses, it is possible to log information detected from between the eyebrows and detect certain emotions for visual information. It can be used.

DESCRIPTION OF SYMBOLS 11 Detection part 12 Light irradiation part 13 Light reception part 15 Control part 16 A / D converter 17 Processing part 18 Display apparatus 20 Information display apparatus 21 Detection part 22 Imaging part 23 Position information acquisition part 24 Additional information processing part 25 A / D conversion Instrument 26 Display 30 Object

Claims (8)

Detecting means for detecting a state between eyebrows of a human who is viewing information;
A display control apparatus comprising: control means for controlling a display state of at least one of the information and additional information added to the information according to a state between eyebrows detected by the detection means.   The display control device according to claim 1, wherein the detection unit is an optical sensor attached to the glasses-type device worn by the person.   The display control device according to claim 2, wherein the glasses-type device is a head-mounted display (HMD). Display means for displaying information;
Detecting means for detecting a state between the eyebrows of a human viewing the information;
An information display device comprising: control means for controlling a display state of at least one of the information and additional information added to the information according to a state between eyebrows detected by the detection means.   The information display apparatus according to claim 4, wherein the control unit controls display of additional information added to the information.   5. The information display device according to claim 4, wherein the control means controls enlargement / reduction of a display state of at least one of the information and the additional information.   The information display device according to claim 4, wherein the control unit controls a reproduction speed of a display state of at least one of the information and the additional information. An information display device that displays augmented reality that is output in a state where additional information added to the image can be recognized on an image in the real space,
Imaging means for capturing an image of the real space;
Display means for displaying the captured image;
Detecting means for detecting a state between human eyebrows;
Position information acquisition means for acquiring position information of the real space;
Based on the acquired position information, at least one of a process of acquiring additional information associated with the position information and a process of generating additional information associated with the position information is performed and detected by the detection means And an additional information processing means for controlling a superimposed display state of the processed additional information on the captured image based on the state between the eyebrows.

Images