WO2017051721A1

WO2017051721A1 - Information processing device, information processing method, and program

Info

Publication number: WO2017051721A1
Application number: PCT/JP2016/076559
Authority: WO
Inventors: 翼塚原; 木村　淳
Original assignee: ソニー株式会社
Priority date: 2015-09-24
Filing date: 2016-09-09
Publication date: 2017-03-30

Abstract

This technology relates to an information processing device, an information processing method and a program capable of preventing operational errors with respect to head-mounted displays (HMD). An input/output control unit controls output with respect to an operation input into an operation accepting unit provided in an enclosure worn on the head of a user, on the basis of information relating to a state of movement of the user. This technology is applicable to display modules which are constituents of HMDs.

Description

Information processing apparatus, information processing method, and program

The present technology relates to an information processing device, an information processing method, and a program, and more particularly, to an information processing device, an information processing method, and a program that can prevent an erroneous operation on an HMD.

In recent years, technology development related to information processing devices worn on the head has been carried out. Such an information processing apparatus is called HMD (Head-Mounted Display). The HMD has a function of providing an image to a user who wears the HMD.

The image display control in the HMD is performed based on, for example, an operation on an input device provided in a frame constituting the HMD. For example, Patent Document 1 discloses an HMD that includes a touch sensor on the side surface of a spectacle-shaped frame.

JP2015-105118A

By the way, when the user wearing the HMD disclosed in Patent Document 1 operates the touch sensor while exercising, there is a risk of erroneous operation.

This technology has been made in view of such a situation, and is intended to prevent erroneous operation on the HMD.

An information processing apparatus according to an aspect of the present technology suppresses reflection of an operation to an operation reception unit provided in a housing attached to the user's head on output based on information on a user's exercise state An output control unit is provided.

An information processing method according to one aspect of the present technology suppresses reflection of an operation to an operation reception unit provided in a housing attached to the user's head on output based on information on the user's exercise state. Including the steps of:

A program according to an aspect of the present technology reflects, in an output, an operation to an operation reception unit provided in a housing attached to the user's head based on information on the user's movement state. A process including the step of suppressing is executed.

In one aspect of the present technology, the reflection of the operation to the operation reception unit provided in the housing mounted on the user's head is suppressed based on the information on the user's motion state.

の一 According to one aspect of the present technology, it is possible to prevent an erroneous operation on the HMD.

It is an external view which shows the structural example of the display module which concerns on this Embodiment. It is a perspective view which shows the structural example of an optical unit. It is sectional drawing which shows the structural example of an optical unit. It is a figure which shows the example of the state with which the display module was mounted | worn by the user. It is a figure which shows the example of an internal structure of a control unit. It is an external view which shows the structural example of a display module. It is a block diagram which shows the function structural example of the display module of 1st Embodiment. It is a flowchart explaining a display process. It is a figure which shows the example of the image displayed on an optical unit. It is a figure which shows the example of the image displayed on an optical unit. It is a figure which shows the example of the image displayed on an optical unit. It is an external view which shows the other structural example of a display module. It is a block diagram which shows the other function structural example of a control unit. It is a flowchart explaining an imaging process. It is a block diagram which shows the function structural example of the display module of 2nd Embodiment. It is a flowchart explaining an input / output control process. It is a figure which shows the example of the image displayed on an optical unit. It is a figure which shows the example of the image displayed on an optical unit. It is a block diagram which shows the other function structural example of 1st Embodiment. It is a block diagram which shows the further another function structural example of 1st Embodiment. It is a block diagram which shows the other function structural example of 2nd Embodiment. It is a block diagram which shows the further another function structural example of 2nd Embodiment. It is a block diagram which shows the function structural example of a computer.

Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be given in the following order.
1. 1. Outline of display module 1st Embodiment (example which controls the output of a touch sensor according to an attachment position)
3. Second embodiment (an example of controlling the output of a touch sensor in accordance with an exercise state)
4). Other

<1. Outline of display module>
FIG. 1 shows a configuration example of a display module according to the present embodiment.

As shown in FIG. 1, the display module 10 and the eyeglass-type frame 20 constitute an HMD (Head-Mounted Display). The display module 10 as an information processing apparatus is attached to the right temple part (right eye side) of the pair of temple parts of the eyeglass-type frame 20 attached to the user's head.

The display module 10 includes a control unit 11, a display unit 12, and an optical unit 13.

The control unit 11 is connected to the display unit 12 and the eyeglass-type frame 20. Specifically, the end of the casing of the control unit 11 in the long side direction is coupled to the display unit 12, and the side surface of the casing of the control unit 11 is connected to the temple portion of the glasses-type frame 20 via a connecting member. The

The control unit 11 incorporates a control board for controlling the operation of the display module 10. Specifically, the control unit 11 includes a control board having a CPU (Central Processing Unit) or a RAM (Random Access Memory).

The touch surface 21 a of the touch sensor 21 is provided on the upper surface of the casing of the control unit 11. As will be described later, a touch surface 21b (not shown) of the touch sensor 21 is provided on the lower surface of the housing of the control unit 11. The

touch surfaces

21a and 21b receive user operations.

When the user is wearing the display module 10 (glasses frame 20), the user cannot visually recognize the

touch surfaces

21a and 21b provided on the housing of the control unit 11. Therefore, the

touch surfaces

21a and 21b are more than the surface of the casing of the control unit 11 on which they are provided so that the user can recognize the positions of the

touch surfaces

21a and 21b by touching the surface of the casing of the control unit 11. Make it indented. The touch surfaces 21a and 21b may be made of a material different from the surface of the housing of the control unit 11 on which the touch surfaces 21a and 21b are provided.

The control unit 11 is connected to the optical unit 13 via the display unit 12.

The display unit 12 connects the control unit 11 and the optical unit 13 and supports the optical unit 13. Specifically, the display unit 12 is coupled to the end of the control unit 11 and the end of the optical unit 13 to fix the optical unit 13. The display unit 12 includes a signal line for communicating data relating to an image provided from the control unit 11 to the optical unit 13 and a display for projecting the image onto the optical unit 13.

The optical unit 13 projects image light to the outside of the display module 10. Specifically, the optical unit 13 sends image light provided from the control unit 11 via the display unit 12 to the outside of the display module 10 through the eyepiece, that is, to the eyes of the user wearing the display module 10. Project.

(Configuration of optical unit)
Here, the configuration of the optical unit 13 will be described in detail with reference to FIGS.

FIG. 2 is a perspective view showing a configuration example of the optical unit 13, and FIG. 3 is a cross-sectional view showing a configuration example of the optical unit 13. FIG. 4 is a diagram illustrating an example of a state in which the display module 10 is mounted on the user.

2 and 3, the optical unit 13 includes a light guide unit 31, a reflection unit 32, and an eyepiece 33.

Image light from the projection unit 41 provided in the display unit 12 is emitted to the light guide unit 31. The projection unit 41 is a display panel that displays an image provided from the control unit 11, and emits image light toward the light guide unit 31 by displaying the image. The projection unit 41 is configured by, for example, an organic EL (Electro-Luminescence) display.

The light guide unit 31 guides the incident image light to the reflection unit 32. The light guide unit 31 is, for example, a columnar member as shown in FIG. As shown in FIG. 3, the light guide unit 31 guides the image light so that the image light projected from the projection unit 41 reaches the reflection unit 32 without leaking outside the display module 10. In addition, the light guide part 31 may be a cylindrical member having a hollow inside, or may be a transparent member that transmits image light.

The reflection unit 32 reflects the reaching image light toward the eyepiece 33. For example, as illustrated in FIG. 3, the reflection unit 32 reflects the image light guided by the light guide unit 31 toward the position of the eyepiece lens 34.

The eyepiece 33 enlarges the image. Specifically, the eyepiece 33 refracts the image light reflected by the reflection unit 32 and enlarges the image related to the image light.

In addition, the optical unit 13 is formed such that the length in the direction in which the light is emitted toward the user's pupil is shorter than the other direction in the region where the light is emitted toward the user's pupil is equal to or shorter than the upper limit of pupil diameter variation. Is done. Specifically, the optical unit 13 is formed so that the width of the optical unit 13 in the short direction is equal to or less than the average pupil diameter.

As shown in FIG. 4, the optical unit 13 is formed such that the length L2 of the width in the short direction of the optical unit 13 is equal to or less than the average pupil diameter L1, for example. Since the human pupil diameter L1 generally varies in the range of about 2 mm to 8 mm, L2 is set to 8 mm or less, for example, 4 mm. Further, in the optical unit 13, the shape of the region where light is emitted may be a circle, an ellipse, or another polygon in addition to a rectangle.

(Configuration of control unit)
Next, the configuration of the control unit 11 will be described in detail with reference to FIG.

FIG. 5 is a cross-sectional view showing an example of the internal configuration of the control unit 11.

As shown in FIG. 5, a touch sensor 21 is provided on the front side of the control unit 11 (the lens part side of the spectacles-type frame 20). The touch surface 21a of the touch sensor 21 is provided on the first surface (upper surface in FIG. 5) of the housing of the control unit 11, and the touch surface 21b is on the opposite side of the first surface of the housing of the control unit 11. It is provided on the second surface (the lower surface in FIG. 5).

The touch sensor 21 is a capacitive touch sensor that detects a capacitance generated between the electrode and the object. The touch surfaces 21a and 21b are configured by arranging a plurality of touch electrodes on each of two substrates provided on the upper surface side and the lower surface side of the housing of the control unit 11. The two substrates constituting each of the touch surfaces 21a and 21b are electrically connected to each other by a flexible flat cable (FFC) 51. With such a configuration, a plurality of touch surfaces can be controlled by one sensor chip, and the cost of the sensor chip can be suppressed.

Note that a sensor chip may be provided for each substrate constituting one touch surface.

Also, since the periphery of the touch electrode needs to be connected to GND, the area (space) sandwiched between the touch surfaces 21a and 21b is shielded. Therefore, as shown in FIG. 5, the wireless antenna 52 is provided on the rear side of the control unit 11 (on the side opposite to the lens portion of the spectacles frame 20).

The wireless antenna 52 performs wireless communication with an external device (for example, a mobile terminal owned by a user) using, for example, Wi-Fi (registered trademark) or Bluetooth (registered trademark).

Note that the display module 10 including the control unit 11 can be connected to any one of the temple portions of the eyeglass-type frame 20.

That is, in the example of FIG. 1, the display module 10 is assumed to be attached to the right temple portion (right eye side) of the temple portions of the eyeglass-type frame 20. Further, as shown in FIG. 6, the display module 10 can be attached to the left temple portion (left eye side) of the temple portions of the eyeglass-type frame 20.

In the example of FIG. 6, the touch surface 21 b of the touch sensor 21 is provided on the upper surface of the housing of the control unit 11, and the touch surface 21 a (not shown) of the touch sensor 21 is disposed on the lower surface of the housing of the control unit 11. Will be provided.

By providing the touch surface of the touch sensor 21 on each of the upper surface and the lower surface of the housing of the control unit 11, the display module 10 can be attached to any of the left and right temple portions of the glasses-type frame 20. The user can operate the touch sensor 21 with the same operability.

<2. First Embodiment (Example of controlling output of touch sensor according to attachment position)>
Hereinafter, the display module 10 according to the first embodiment of the present technology will be described.

(Example of functional configuration of display module)
FIG. 7 illustrates a functional configuration example of the display module 10 according to the first embodiment of the present technology.

The display module 10 shown in FIG. 7 includes operation reception units 111a and 111b, a mounting state determination unit 112, a display control unit 113, and a display unit 114.

The operation reception units 111 a and 111 b correspond to the touch surfaces 21 a and 21 b of the touch sensor 21. The operation reception units 111a and 111b receive user operations such as taps, long taps, and swipes. The operation reception units 111a and 111b supply a signal corresponding to the received user operation to the display control unit 113.

The mounting state determination unit 112 determines the mounting state of the housing of the control unit 11. Specifically, the wearing state determination unit 112 determines which of the pair of temple portions of the eyeglass-type frame 20 the control unit 11 (housing) is connected to. The wearing state determination unit 112 supplies a signal representing the determination result to the display control unit 113.

For example, the wearing state determination unit 112 is configured to include an acceleration sensor, and determines which of the temple units is connected by determining the posture of the housing based on the output of the acceleration sensor. In addition, the wearing state determination unit 112 is configured to include a gyro sensor, and determines the position of the housing based on the output of the gyro sensor, thereby determining which of the temple units is connected. May be.

Furthermore, the wearing state determining unit 112 may determine which of the temple units is connected by performing near field communication such as NFC (Near Field Communication) with the glasses-type frame 20. Further, the mounting state determination unit 112 may determine which of the temple units is connected based on the positional relationship between the control unit 11 and the optical unit 13.

Furthermore, the wearing state determination unit 112 may determine which of the temple units is connected based on information preset by the user. For example, in the setting mode in which various settings of the display module 10 are performed, the user can select which of the temple parts to connect to, so that the wearing state determination unit 112 can determine whether the temple part has You may make it discriminate | determine which is connected.

The display control unit 113 is configured as an output control unit that controls output according to an operation on the operation receiving units 111a and 111b, depending on the determined mounting state of the control unit 11 (housing). Specifically, the display control unit 113 controls the display of an image on the display unit 114 based on signals from the operation receiving units 111a and 111b and the wearing state determination unit 112.

The display unit 114 corresponds to the display unit 12 and the optical unit 13. As shown in FIGS. 1 and 6, the display unit 114 may be provided in the vicinity of the lens unit of the spectacles type frame 20 or may be provided in the lens unit of the spectacles type frame 20 itself. Good.

(Display module display processing)
Next, display processing of the display module 10 will be described with reference to the flowchart of FIG. The process of FIG. 8 is started when, for example, the display module 10 (the housing of the control unit 11) is connected to one of the temple parts of the eyeglass-type frame 20.

In step S11, the wearing state determination unit 112 determines whether the housing of the control unit 11 is connected to the left or right temple unit. The wearing state determination unit 112 supplies a signal representing the determination result to the display control unit 113.

In step S12, the display control unit 113 displays a screen on the display unit 114 in a direction corresponding to which of the left and right temple units is connected based on the signal from the mounting state determination unit 112. Image light projected from the optical unit 13 when the housing of the control unit 11 is connected to the right temple (FIG. 1) and when connected to the left temple (FIG. 6). Is upside down from the user's perspective. Therefore, the display control unit 113 switches the orientation of the image displayed on the display unit 114 in the vertical direction depending on which of the left and right temple units is connected to the housing. Further, in this case, the output of the acceleration sensor or gyro sensor constituting the mounting state determination unit 112 is treated as being inverted depending on which of the left and right temple portions is connected to the case. Also become. Specifically, the output of the acceleration sensor or the gyro sensor itself may be reversed, or the display control unit 113 may recognize that the output of the acceleration sensor or the gyro sensor is reversed.

At this time, the display control unit 113 may display on the display unit 114 information indicating which of the left and right temple units the casing of the control unit 11 is connected to.

For example, when the case of the control unit 11 is connected to the right temple part, as shown in the upper part of FIG. 9, the right side temple is displayed on the upper right side of the screen displayed on the display unit 114. “Right-side mode” wording 131 indicating that the unit is connected to the unit is displayed.

Further, when the housing of the control unit 11 is connected to the left temple portion in the upper state of FIG. 9, the housing displayed on the upper left of the screen displayed on the display unit 114 as shown in the lower portion of FIG. The wording 132 of “left side mode” indicating that the body is connected to the left temple part is displayed.

Note that the screen shown in FIG. 9 is an example of a menu screen of an application related to display provided by the display module 10. In the example of FIG. 9, icons representing the menu B, the menu C, and the menu D are displayed. The icon representing the menu C is displayed larger than the other icons in the center of the menu screen and is in a selectable state. The icon displayed on the menu screen is scroll-displayed in the horizontal direction (left-right direction) by a user operation on the touch sensor 21.

Referring back to the flowchart of FIG. 8, the display control unit 113 determines whether or not a touch surface facing down among the touch surfaces 21 a and 21 b of the touch sensor 21 has been operated. Specifically, when the housing of the control unit 11 is connected to the right temple portion (FIG. 1), it is determined whether or not the touch surface 21b has been operated. On the other hand, when the housing of the control unit 11 is connected to the left temple portion (FIG. 6), it is determined whether or not the touch surface 21a has been operated.

If it is determined in step S13 that the touch surface facing downward has been operated, the process proceeds to step S14.

In step S14, the display control unit 113 performs screen display on the display unit 114 in accordance with a user operation on the touch surface facing downward.

For example, when the menu screen shown in the upper part of FIG. 10 is displayed on the display unit 114, a swipe operation from the rear to the front is performed on the touch surface facing downward. The screen scrolls to the left. As a result, as shown in the lower part of FIG. 10, the icon representing the menu C displayed on the right side of the menu screen is displayed largely in the center of the menu screen.

When the menu screen shown in the upper part of FIG. 11 is displayed on the display unit 114, when a swipe operation from the front to the rear is performed on the touch surface facing downward, the menu is displayed. The screen scrolls to the right. As a result, as shown in the lower part of FIG. 11, the icon representing the menu B displayed on the left side of the menu screen is displayed largely at the center of the menu screen.

On the other hand, if it is determined in step S13 that the touch surface facing downward is not operated, the display control unit 113 does nothing until the touch surface facing downward is operated. Therefore, even if the touch surface facing upward is operated, the menu screen shown in the upper part of FIGS. 10 and 11 does not scroll.

That is, the display control unit 113 suppresses the reflection of the operation on the touch surface facing upward on the output. Specifically, the display control unit 113 invalidates the operation on the touch surface facing upward. In other words, even if the touch surface facing upward is operated, the display control unit 113 does not determine that there is an operation or does not perform display control after determining that there is an operation. Thereby, the user's operation on the touch surface facing upward is not reflected on the screen displayed on the display unit 114. In addition, the display control unit 113 turns off the power supply to the electrode corresponding to the touch surface facing upward among the touch surfaces of the touch sensor 111, so that the user's operation to the touch surface facing upward can be performed. You may make it not reflect on the screen currently displayed on the display part 114. FIG.

According to the above processing, even when the display module is connected to any of the left and right temple parts of the spectacles-type frame, one of the two touch surfaces corresponds to an operation on the touch surface facing downward. Screen display is performed. In other words, even if the display module is connected to any of the left and right temple portions of the eyeglass-type frame, the user can realize the same screen display by the same operation in each case. As a result, the operability of the HMD can be further improved.

In the above, the display control unit 113 disables the operation to the operation reception unit 111a or 111b so that the user operation is not reflected on the screen displayed on the display unit 114. Not limited to this, the user's operation may not be reflected on the screen displayed on the display unit 114 but may be made difficult to be reflected.

For example, the display control unit 113 reduces the gain of the output amount with respect to the operation input amount of the operation receiving unit 111a or 111b based on the mounting state of the housing of the control unit 11.

Further, the display control unit 113 may tighten the detection conditions for the operation to the operation receiving unit 111a or 111b based on the mounting state of the housing of the control unit 11. For example, based on the mounting state of the casing of the control unit 11, the capacitance value serving as a reference for detecting a user operation (tap or swipe) on the touch surface facing upward is increased.

Furthermore, the display control unit 113 may reduce the range in which an operation to the operation receiving unit 111a or 111b can be detected based on the mounting state of the housing of the control unit 11.

Further, the display control unit 113 may lower the detection sensitivity of the operation to the operation receiving unit 111a or 111b based on the mounting state of the housing of the control unit 11.

In the above description, the operation reception units 111a and 111b correspond to the touch surfaces 21a and 21b of the touch sensor 21, but may correspond to other configurations. For example, the operation receiving units 111a and 111b may be physical input devices such as buttons and dials. The operation receiving units 111a and 111b may be sensors that detect a user's gesture or line of sight by detecting visible light, infrared light, radio waves, sound waves, or the like.

(Modification)
Note that, as an operation on the touch surface facing downward, in addition to the swipe operation in the front-rear direction, a swipe operation in the left-right direction, that is, a planar operation on the touch surface may be accepted. Further, a direction in which the finger is separated from the touch surface may be detected, and a three-dimensional operation on the touch surface may be accepted.

In the above description, two touch surfaces are provided on each of the upper surface and the lower surface of the housing of the control unit 11. However, only one touch surface is provided on one surface of the housing of the control unit 11. You may do it. In this case, the housing of the control unit 11 includes, for example, left and right temple portions so that the front and rear directions of the display unit 12 and the optical unit 13 are switched so that the surface on which the touch surface is provided is always directed downward. It shall have a structure connectable to.

In such a configuration, the relative finger of the touch surface to the touch surface is determined depending on whether the housing of the control unit 11 is connected to the left temple portion or the right temple portion. The direction of movement is opposite to the direction of finger movement detected by the touch surface.

Therefore, in such a configuration, the display control unit 111 switches the output (screen display) according to the operation on the touch surface so as to be reversed based on the mounting state of the housing of the control unit 11. For example, the display control unit 111 reverses the display control for the signal from the touch surface depending on whether the housing of the control unit 11 is connected to the right temple unit or the left temple unit. .

For example, when the casing of the control unit 11 is connected to the right temple part and the menu screen shown in the upper part of FIG. Suppose that a swipe operation toward is performed. At this time, the direction of the finger movement detected by the touch surface is the direction from the far side to the near side from the connecting portion with the temple portion. In this case, the menu screen scrolls to the left. As a result, as shown in the lower part of FIG. 10, the icon representing the menu C displayed on the right side of the menu screen is displayed largely in the center of the menu screen.

On the other hand, in the state where the case of the control unit 11 is connected to the left temple part and the menu screen shown in the upper part of FIG. The same swipe operation from right to left is performed. At this time, the direction of the movement of the finger detected by the touch surface is opposite to the above-described example, and the direction is from the side closer to the far side from the connecting portion to the temple portion. Even in this case, the menu screen scrolls to the left. As a result, as shown in the lower part of FIG. 10, the icon representing the menu C displayed on the right side of the menu screen is displayed largely in the center of the menu screen.

In the above-described example, the display control unit 111 inverts the signal from the touch surface depending on whether the housing of the control unit 11 is connected to the right temple unit or the left temple unit. Thus, the screen display may be reversed.

Furthermore, in the above description, it is assumed that only the operation on the touch surface facing downward among the two touch surfaces provided on the upper surface and the lower surface of the casing of the control unit 11 is accepted. Only an operation on the touch surface may be accepted, or an operation on both touch surfaces may be accepted. When operations on both touch surfaces are accepted, for example, a predetermined command may be started by simultaneously performing a tap operation or a swipe operation on both touch surfaces.

(Other display module configuration examples)
FIG. 12 is an external view showing another configuration example of the display module according to the first embodiment.

The display module 10 shown in FIG. 12 further includes a camera 151 in addition to the configuration of the display module 10 shown in FIG.

The camera 151 is provided on the surface of the housing of the control unit 11 on the surface side of the eyeglass-type frame 20 so that the lens is exposed. The camera 151 captures an image of the visual field direction of the eyeglass-type frame 20.

(Example of functional configuration of display module)
FIG. 13 shows a functional configuration example of the display module 10 shown in FIG.

The display module 10 shown in FIG. 13 further includes an imaging unit 171 and a recording unit 172 in addition to the configuration of the display module 10 shown in FIG.

The imaging unit 171 corresponds to the camera 151. The imaging unit 171 supplies the captured image data to the display control unit 113. The display control unit 113 displays an image on the display unit 114 based on the data from the imaging unit 171.

In addition, the imaging unit 171 supplies the captured image data to the recording unit 172. That is, the recording unit 172 records the image captured by the imaging unit 171.

(Image capture processing of display module)
Next, imaging processing of the display module 10 of FIG. 13 will be described with reference to the flowchart of FIG. The process of FIG. 14 is started when the display module 10 (the housing of the control unit 11) is connected to one of the temple parts of the eyeglass-type frame 20.

In step S31, the mounting state determination unit 112 determines whether the housing (control unit 11) is connected to the left or right temple unit. The wearing state determination unit 112 supplies a signal representing the determination result to the display control unit 113.

In step S <b> 32, the display control unit 113 displays a live view, a so-called live view, in a direction corresponding to which of the left and right temples is connected to the display unit based on the signal from the mounting state determination unit 112. 114. In the case where the casing of the control unit 11 is connected to the temple part on the right side and the case where it is connected to the temple part on the left side, the image light projected from the optical unit 13 has a top-and-bottom direction as viewed from the user. Vice versa. Accordingly, the display control unit 113 switches the top-to-bottom direction of the through image displayed on the display unit 114 depending on which of the left and right temple units the casing is connected to.

In step S <b> 33, the display control unit 113 determines whether or not the touch surface facing down of the touch surfaces 21 a and 21 b of the touch sensor 21 has been tapped, for example. Specifically, when the housing of the control unit 11 is connected to the right temple, it is determined whether or not the touch surface 21b has been operated. On the other hand, when the housing of the control unit 11 is connected to the left temple, it is determined whether or not the touch surface 21a has been operated.

If it is determined in step S33 that the touch surface facing downward has been operated, the process proceeds to step S34.

In step S34, the imaging unit 171 causes the recording unit 172 to record a still image captured when the touch surface facing downward is operated. This still image is also an image oriented according to whether the housing is connected to the left or right temple part.

On the other hand, if it is determined in step S33 that the touch surface facing downward is not operated, the still image is not recorded until the touch surface facing downward is operated. Therefore, even if the touch surface facing upward is operated, a still image is not recorded.

According to the above processing, even when the display module is connected to any of the left and right temple portions of the spectacle-shaped frame, according to the operation on the touch surface facing downward of the two touch surfaces. A still image is recorded. In other words, even when the display module is connected to any of the left and right temple portions of the eyeglass-type frame, the user can realize imaging by the same operation in each case. As a result, the operability of the HMD can be further improved.

<3. Second Embodiment (Example of Controlling Output of Touch Sensor According to Movement State)>
Hereinafter, the display module 10 according to the second embodiment of the present technology will be described.

(Example of functional configuration of display module)
FIG. 15 illustrates a functional configuration example of the display module 10 according to the second embodiment of the present technology.

15 includes an operation reception unit 211, an exercise state acquisition unit 212, a display control unit 213, and a display unit 214.

The operation reception unit 211 corresponds to the touch surfaces 21 a and 21 b of the touch sensor 21. The operation reception unit 211 receives user operations such as tap, long tap, and swipe. The operation reception unit 211 supplies a signal corresponding to the received user operation to the display control unit 213.

In this example, it does not matter which of the touch surfaces 21a and 21b corresponding to the operation reception unit 211 is operated. That is, as in the first embodiment, only the touch surface facing downward may accept an operation, or any touch surface may accept an operation.

The exercise state acquisition unit 212 acquires information regarding the exercise state of the user wearing the display module 10. Specifically, the exercise state acquisition unit 212 acquires the user's exercise amount (strength of exercise) as information on the exercise state. The intensity of exercise is calculated by, for example, biological sensing (sensing of biological information such as heart rate, sweating, body temperature, blood pressure, myoelectric potential, brain wave, respiration, etc.). Further, the intensity of exercise may be calculated based on the movement speed of the user and the result of action recognition such as whether the user is stationary, walking, or running. The exercise state acquisition unit 212 supplies a signal representing information about the acquired exercise state to the display control unit 113.

For example, the exercise state acquisition unit 212 is configured to include an acceleration sensor, and acquires the output of the acceleration sensor as information related to the user's exercise state. Further, the exercise state acquisition unit 212 may be configured to include a GPS (Global Positioning System) sensor, and the output of the GPS sensor may be acquired as information on the user's exercise state.

The display control unit 213 controls display of an image on the display unit 214.

Further, the display control unit 213 is configured as an output control unit that controls an output to the operation receiving unit 211 based on the acquired information on the user's exercise state. Specifically, the display control unit 213 controls an output for an operation on an image displayed on the display unit 214 based on a signal from the exercise state acquisition unit 212.

The display unit 214 corresponds to the display unit 12 and the optical unit 13. As shown in FIGS. 1 and 6, the display unit 214 may be provided in the vicinity of the lens unit of the glasses-type frame 20 or may be provided in the lens unit of the glasses-type frame 20 itself. Good.

(Display module input / output control processing)
Next, display processing of the display module 10 will be described with reference to the flowchart of FIG. The process of FIG. 16 is started in a state in which the eyeglass frame 20 to which the display module 10 is connected is attached to the user and a predetermined screen is displayed on the display unit 214.

In step S71, the exercise state acquisition unit 212 acquires information related to the user's exercise state. The exercise state acquisition unit 212 supplies a signal representing information related to the user's exercise state to the display control unit 213.

In step S72, the display control unit 213 determines whether or not the user's exercise state is a predetermined exercise state based on the signal from the exercise state acquisition unit 212. Specifically, the display control unit 213 determines whether or not the amount of exercise exceeds a predetermined amount (predetermined level) based on the amount of exercise (intensity of exercise) of the user. For example, the display control unit 213 can determine that the amount of exercise of the user exceeds a predetermined amount when the movement speed of the user exceeds a predetermined speed. Further, the display control unit 213 can determine that the amount of exercise of the user exceeds a predetermined amount when the user is traveling. Furthermore, when the value indicated by the biological information, for example, the value of the heartbeat is higher than the predetermined value, the display control unit 213 determines that the user's exercise amount exceeds the predetermined amount based on the biological information obtained by the biological sensing. Can be determined. Furthermore, the display control unit 213 calculates the vibration of the display module 10 (control unit 11) from the output of the acceleration sensor included in the motion state acquisition unit 212, and when the period of the vibration is shorter than a predetermined time, When the amplitude of vibration is larger than a predetermined amount, it can be determined that the user's momentum exceeds a predetermined amount.

If it is determined in step S72 that the user's exercise state is a predetermined exercise state, that is, the user's exercise amount (intensity of exercise) exceeds a predetermined amount (predetermined level), the process proceeds to step S73. move on.

In step S <b> 73, the display control unit 213 displays an operation on the operation reception unit 211 on the display unit 214 by controlling an output to the operation reception unit 211 with respect to the screen displayed on the display unit 214. Suppress the reflection on the screen. Specifically, the operation to the operation reception unit 211 is invalidated. In other words, even if the operation receiving unit 211 is operated, the display control unit 213 does not determine that there is an operation or does not perform display control after determining that there is an operation. Thereby, the user's operation is not reflected on the screen displayed on the display unit 214.

At this time, the display control unit 213 may display information indicating a change in the exercise state of the user on the screen displayed on the display unit 214.

For example, in the state where the menu screen shown in the upper part of FIG. 17 is displayed, when the intensity of the user's exercise exceeds a predetermined level, as shown in the lower part of FIG. A word 231 of “input restriction mode” indicating that the user's operation is restricted is displayed.

Further, the color of the indicator 251 displayed at the upper right of the menu screen shown in FIG. 18 may be changed according to the user's exercise state. Specifically, in a state where the intensity of the user's exercise does not exceed a predetermined level, the color of the indicator 251 is, for example, white or green as shown in the upper part of FIG. When the intensity of the user's exercise exceeds a predetermined level, the color of the indicator 251 changes to, for example, black or red as shown in the lower part of FIG.

Returning to the flowchart of FIG. 16, when it is determined in step S72 that the user's exercise state is not the predetermined exercise state, that is, the intensity of the user's exercise does not exceed the predetermined level, the process proceeds to step S71. Returning, the processes of steps S71 and S72 are repeated.

According to the above processing, when the intensity of the user's exercise exceeds a predetermined level, the operation on the touch surface of the touch sensor is invalidated. Therefore, even if the user wearing the spectacle-type frame 20 to which the display module 10 is connected operates the touch sensor while exercising, there is no risk of incorrect operation, and erroneous operation on the HMD is eliminated. It becomes possible to prevent.

In the above, even if the operation receiving unit 211 is operated, the display control unit 213 does not determine that there is an operation or does not perform display control after determining that there is an operation. The user operation is no longer reflected on the screen displayed on the display unit 214. In addition to this, the display control unit 213 may turn off the power supply to the operation reception unit 211 so that the user's operation is not reflected on the screen displayed on the display unit 214.

Further, the user operation may not be reflected on the screen displayed on the display unit 214 but may be made difficult to reflect.

For example, the display control unit 213 reduces the gain of the output amount with respect to the operation input amount of the operation receiving unit 211 based on the user's exercise state. For example, when the user's exercise intensity exceeds a predetermined level, the scroll amount of the menu screen relative to the finger movement amount in the swipe operation to the operation reception unit 211 is normal (the user's exercise intensity is predetermined). (If the level is not exceeded).

Further, the display control unit 213 may tighten the detection conditions for the operation to the operation receiving unit 211 based on the user's exercise state. For example, based on the user's exercise state, the capacitance value serving as a reference for detecting a user operation (tap or swipe) in the operation reception unit 211 may be increased. Thereby, when the intensity of the user's exercise exceeds a predetermined level, the detection sensitivity of the operation reception unit 211 itself can be lowered.

Furthermore, the display control unit 213 may reduce the range in which an operation to the operation receiving unit 211 can be detected based on the user's exercise state.

Further, the display control unit 213 may reduce the detection sensitivity of the operation to the operation receiving unit 211 based on the user's exercise state.

Furthermore, the display control unit 213 may cause the operation receiving unit 211 to output an operation after a predetermined operation based on the user's exercise state. For example, when the intensity of the user's exercise exceeds a predetermined level, an output for a normal tap or swipe is made after a long tap for a predetermined time or longer is given to the operation reception unit 211. As a result, the user performs an operation on the operation receiving unit 211 after suppressing the exercise, and as a result, an erroneous operation can be prevented.

Further, the display control unit 213 may invalidate a steady operation on the operation receiving unit 211 based on the user's exercise state. For example, when the intensity of the user's exercise exceeds a predetermined level, the continuous touch on the operation reception unit 211 is ignored. Thereby, even if the decoration of the helmet worn by the user, the user's own hair, etc. are always touching the touch surface of the touch sensor 21, it is possible to prevent erroneous operations caused by the touch. .

In the above, the output for the operation to the operation reception unit 211 is controlled based on the amount of exercise of the user, but the output for the operation is further controlled based on the direction of movement of the user. Good.

For example, when the user's movement direction is a predetermined direction and the user's movement amount exceeds a predetermined amount, the output for the operation is controlled. For example, when the user is running or riding a bicycle, the operation of the operation reception unit 211 is more difficult when the user is running with a lot of vertical movement. Therefore, even when the amount of exercise is the same, when the amount of exercise in the vertical direction exceeds a predetermined amount, reflection of the operation on the output is suppressed. Thereby, it becomes possible to prevent an erroneous operation by a user who is running.

In the above description, the operation reception unit 211 corresponds to the touch surfaces 21a and 21b of the touch sensor 21, but may correspond to other configurations. For example, the operation reception unit 211 may be a physical input device such as a button or a dial. In addition, the operation reception unit 211 may be sensors that detect a user's gesture or line of sight by detecting visible light, infrared light, radio waves, sound waves, or the like.
<4. Other>

In addition, you may make it combine the structure which concerns on the 1st and 2nd embodiment mentioned above, respectively.

Further, the first embodiment described above can have the following configuration.

For example, as shown in FIG. 19, the HMD 410 including the operation receiving units 111a and 111b, the wearing state determining unit 112, and the display unit 114 and the information processing apparatus 420 including the display control unit 113 are configured as separate units. You may make it mutually perform wireless communication using wired communication or Wi-Fi (trademark), Bluetooth (trademark), etc. mutually. In this case, the information processing apparatus 420 is configured as a mobile terminal such as a smartphone.

Furthermore, as shown in FIG. 20, the HMD 410 and the information processing apparatus 420 may be connected via a network 430 such as the Internet or an intranet. In this case, the information processing apparatus 420 is configured as a server on a network, for example.

Further, the second embodiment described above can have the following configuration.

For example, as shown in FIG. 21, an HMD 510 including an operation receiving unit 211, an exercise state acquiring unit 212, and a display unit 214 and an information processing device 520 including a display control unit 213 are configured as separate units, Wired communication or wireless communication using Wi-Fi (registered trademark), Bluetooth (registered trademark), or the like may be performed. In this case, the information processing device 520 is configured as a mobile terminal such as a smartphone.

Furthermore, as shown in FIG. 22, the HMD 510 and the information processing apparatus 520 may be connected via a network 530 such as the Internet or an intranet. In this case, the information processing apparatus 520 is configured as a server on a network.

In the above, the display module 10 as the information processing apparatus of the present technology is configured to include the touch sensor and the display unit, but may be configured not to include the display unit, or vice versa. The touch sensor may not be included.

In the above, the display method of the display module 10 is such that the length in the direction in which the length is shorter than the other direction of the region where the light is emitted toward the pupil of the user is equal to or less than the upper limit of the pupil diameter variation. That is, a method using an eyepiece optical system, that is, the user's pupil is divided into a portion where light related to an external image is incident and a portion where light related to an electronically generated image is incident, and an external image In the above description, the generated image is superposed optically (pupil division method). However, the display method of the display module 10 may be a prism method, a hologram method, or other so-called see-through type or see-around type display method.

That is, the present technology can be applied to a transmissive display type or video see-through type HMD.

Furthermore, the present technology may be applied to an HMD other than the glasses type, for example, a hat type HMD, or may be applied to other wearable terminals such as a wristwatch type or a wristband type.

The series of processes described above can be executed by hardware or software. When a series of processing is executed by software, a program constituting the software is installed in the computer. Here, the computer includes, for example, a general-purpose personal computer capable of executing various functions by installing a computer incorporated in dedicated hardware and various programs.

FIG. 23 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processing by a program.

In the computer, a CPU 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are connected to each other by a bus 904.

An input / output interface 905 is further connected to the bus 904. An input unit 906, an output unit 907, a storage unit 908, a communication unit 909, and a drive 910 are connected to the input / output interface 905.

The input unit 906 includes a keyboard, a mouse, a microphone, and the like. The output unit 907 includes a display, a speaker, and the like. The storage unit 908 includes a hard disk, a nonvolatile memory, and the like. The communication unit 909 includes a network interface or the like. The drive 910 drives a removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer configured as described above, the CPU 901 loads the program stored in the storage unit 908 to the RAM 903 via the input / output interface 905 and the bus 904 and executes the program, for example. Is performed.

The program executed by the computer (CPU 901) can be provided by being recorded on a removable medium 911 as a package medium, for example. The program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

In the computer, the program can be installed in the storage unit 908 via the input / output interface 905 by attaching the removable medium 911 to the drive 910. The program can be received by the communication unit 909 via a wired or wireless transmission medium and installed in the storage unit 908. In addition, the program can be installed in the ROM 902 or the storage unit 908 in advance.

The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

Moreover, this technique can take the following structures.
(1)
An information processing apparatus comprising: an output control unit that suppresses reflection of an operation to an operation receiving unit provided in a housing mounted on the user's head based on information related to a user's exercise state.
(2)
The information processing apparatus according to (1), wherein the output control unit invalidates an operation to the operation receiving unit based on information on the exercise state of the user.
(3)
The information processing apparatus according to (2), wherein the output control unit invalidates an operation to the operation receiving unit based on the amount of exercise of the user.
(4)
The information processing apparatus according to (3), wherein the output control unit invalidates an operation to the operation receiving unit when the amount of exercise of the user exceeds a predetermined amount.
(5)
The information processing apparatus according to (4), wherein the output control unit invalidates an operation to the operation receiving unit when a moving speed of the user exceeds a predetermined speed.
(6)
The information processing apparatus according to (3), wherein the output control unit invalidates an operation to the operation receiving unit based on an amount of exercise and an exercise direction of the user.
(7)
The information processing apparatus according to (1), wherein the output control unit decreases a gain of an output amount with respect to an operation input amount to the operation receiving unit based on information on the user's exercise state.
(8)
The information processing apparatus according to (1), wherein the output control unit tightens a condition for detecting an operation to the operation receiving unit based on information related to the user's exercise state.
(9)
The information processing apparatus according to (1), wherein the output control unit reduces a range in which an operation to the operation receiving unit can be detected based on information on the user's exercise state.
(10)
The information processing apparatus according to (1), wherein the output control unit lowers detection sensitivity of an operation to the operation receiving unit based on information on the user's exercise state.
(11)
The information processing apparatus according to (1), wherein the output control unit causes the operation receiving unit to output an operation after a predetermined operation based on information on the user's exercise state.
(12)
The information processing apparatus according to (1), wherein the output control unit invalidates a steady operation on the operation receiving unit based on information related to the user's exercise state.
(13)
The information processing apparatus according to any one of (1) to (12), wherein the housing is connected to a glasses-type frame that is mounted on the head of the user.
(14)
It further comprises a display unit provided in or near the lens unit of the spectacles type frame,
The information processing apparatus according to (13), wherein the output control unit suppresses reflection of an operation on the operation receiving unit on display on the display unit based on information on the exercise state of the user.
(15)
The information processing apparatus according to (14), wherein the output control unit causes the display unit to display information indicating a change in the exercise state of the user.
(16)
The information processing apparatus according to (1), wherein the operation reception unit includes a touch surface of a touch sensor.
(17)
The information processing apparatus according to (16), wherein the touch surface is formed to be recessed from a surface of the housing on which the touch surface is provided.
(18)
The information processing apparatus according to (16), wherein the touch surface is made of a material different from a surface on which the touch surface of the housing is provided.
(19)
An information processing method including a step of suppressing, on an output, an operation to an operation receiving unit provided in a housing attached to the user's head based on information on a user's exercise state.
(20)
On the computer,
A program for executing a process including a step of suppressing the reflection of an operation to an operation receiving unit provided in a housing attached to the user's head on an output based on information on the user's exercise state .

10 display module, 11 control unit, 12 display unit, 13 optical unit, 20 glasses-type frame, 21 touch sensor, 21a, 21b touch surface, 111a, 111b operation accepting unit, 112 wearing state discriminating unit, 113 display control unit, 114 Display unit, 151 camera, 171 imaging unit, 172 recording unit, 211 operation accepting unit, 212 exercise state acquisition unit, 213 display control unit, 214 display unit

Claims

An information processing apparatus comprising: an output control unit that suppresses reflection of an operation to an operation receiving unit provided in a housing mounted on the user's head based on information related to a user's exercise state.
The information processing apparatus according to claim 1, wherein the output control unit invalidates an operation to the operation receiving unit based on information related to the user's exercise state.
The information processing apparatus according to claim 2, wherein the output control unit invalidates an operation to the operation receiving unit based on an amount of exercise of the user.
The information processing apparatus according to claim 3, wherein the output control unit invalidates an operation to the operation reception unit when the amount of exercise of the user exceeds a predetermined amount.
The information processing apparatus according to claim 4, wherein the output control unit invalidates an operation to the operation receiving unit when the moving speed of the user exceeds a predetermined speed.
The information processing apparatus according to claim 3, wherein the output control unit invalidates an operation to the operation receiving unit based on an amount of exercise and an exercise direction of the user.
The information processing apparatus according to claim 1, wherein the output control unit decreases a gain of an output amount with respect to an operation input amount to the operation receiving unit based on information regarding the exercise state of the user.
The information processing apparatus according to claim 1, wherein the output control unit tightens a condition for detecting an operation to the operation receiving unit based on information related to the user's exercise state.
The information processing apparatus according to claim 1, wherein the output control unit reduces a range in which an operation to the operation receiving unit can be detected based on information on the user's exercise state.
The information processing apparatus according to claim 1, wherein the output control unit lowers detection sensitivity of an operation to the operation receiving unit based on information related to the user's exercise state.
The information processing apparatus according to claim 1, wherein the output control unit causes an output for an operation after a predetermined operation to the operation receiving unit to be performed based on information on an exercise state of the user.
The information processing apparatus according to claim 1, wherein the output control unit invalidates a steady operation on the operation receiving unit based on information related to the user's exercise state.
The information processing apparatus according to claim 1, wherein the housing is connected to a glasses-type frame that is worn on the head of the user.
It further comprises a display unit provided in or near the lens unit of the spectacles type frame,
The information processing apparatus according to claim 13, wherein the output control unit suppresses reflection of an operation on the operation receiving unit on a display on the display unit based on information on the exercise state of the user.
The information processing apparatus according to claim 14, wherein the output control unit causes the display unit to display information indicating a change in the exercise state of the user.
The information processing apparatus according to claim 1, wherein the operation reception unit includes a touch surface of a touch sensor.
The information processing apparatus according to claim 16, wherein the touch surface is formed to be recessed from a surface of the housing where the touch surface is provided.
The information processing apparatus according to claim 16, wherein the touch surface is made of a material different from a surface on which the touch surface of the housing is provided.
An information processing method including a step of suppressing, on an output, an operation to an operation receiving unit provided in a housing attached to the user's head based on information on a user's exercise state.
On the computer,
A program for executing a process including a step of suppressing the reflection of an operation to an operation receiving unit provided in a housing attached to the user's head on an output based on information on the user's exercise state .