JP2018160249A - Head-mount display system, head-mount display, display control program, and display control method - Google Patents

Head-mount display system, head-mount display, display control program, and display control method Download PDF

Info

Publication number
JP2018160249A
JP2018160249A JP2018092931A JP2018092931A JP2018160249A JP 2018160249 A JP2018160249 A JP 2018160249A JP 2018092931 A JP2018092931 A JP 2018092931A JP 2018092931 A JP2018092931 A JP 2018092931A JP 2018160249 A JP2018160249 A JP 2018160249A
Authority
JP
Japan
Prior art keywords
head
display
input
user
mounted display
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
JP2018092931A
Other languages
Japanese (ja)
Inventor
洋輔 栗原
Yosuke Kurihara
洋輔 栗原
Original Assignee
株式会社ソニー・インタラクティブエンタテインメント
Sony Interactive Entertainment LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社ソニー・インタラクティブエンタテインメント, Sony Interactive Entertainment LLC filed Critical 株式会社ソニー・インタラクティブエンタテインメント
Priority to JP2018092931A priority Critical patent/JP2018160249A/en
Publication of JP2018160249A publication Critical patent/JP2018160249A/en
Application status is Pending legal-status Critical

Links

Images

Abstract

PROBLEM TO BE SOLVED: To improve convenience of a user of a head-mount display.SOLUTION: A head-mount display 100 is a head-mount display that can be worn on the head of a user, and comprises: a display that is provided on a surface to be inside the head-mount display 100 when the head-mount display is worn on the head of the user; and a display control part that acquires data of an image to be created on the basis of an operation input from an input device 20 held by the user, and displays the acquired data of the image on the display.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display control technique, and more particularly to a head mounted display system, a head mounted display, a display control program, and a display control method.

  A head-mounted display connected to a game machine is attached to the head, and a game is played by operating a controller or the like while viewing a screen displayed on the head-mounted display. In a normal stationary display connected to a game machine, the user's field of view extends beyond the display screen, so it may not be possible to concentrate on the display screen or lack of immersion in the game. There is. On the other hand, when the head mounted display is mounted, since the user does not see any video other than the video displayed on the head mounted display, the feeling of immersion in the video world is increased and the entertainment property of the game is further enhanced.

  The present inventor has recognized that more user-friendly input technology is necessary in order to allow more user groups to enjoy a game using a head-mounted display.

  In order to solve the above problems, a head-mounted display according to an aspect of the present invention is a head-mounted display that can be mounted on a user's head, and the inner side when the head-mounted display is mounted on the user's head. An input capable of detecting that a part of the user's body is in contact with or close to the display device provided on the surface and the surface provided on the outer surface when the head mounted display is mounted on the user's head. Acquire image data generated based on input information indicating that a part of the user's body is in contact with or close to the apparatus and detected by the input device, and display the acquired image data on the display device A display control unit.

  Another aspect of the present invention is a display control method. In this method, the acquisition unit is provided on a surface that is on the outer side when a head mounted display that can be mounted on the user's head is mounted on the user's head. A step of acquiring data of an image generated based on input information detected by an input device capable of detecting that and indicating that a part of the user's body has touched or approached, and the display control unit includes an acquisition unit And displaying the image data acquired on the display device provided on the inner surface when the head mounted display is mounted on the user's head.

  Yet another embodiment of the present invention is a display system. This display system includes a head-mounted display that can be worn on the user's head, and an image processing device that generates an image to be displayed on the head-mounted display. The head mounted display has a display device provided on the inner surface when the head mounted display is mounted on the user's head, and an outer surface when the head mounted display is mounted on the user's head. Based on the provided input device that can detect that a part of the user's body is in contact with or in proximity, and input information that is detected by the input device and indicates that the part of the user's body is in contact or in proximity A display control unit configured to acquire data of the generated image from the image processing apparatus and display the acquired image data on the display device. The image processing apparatus includes an image generation unit that acquires input information and generates image data based on the input information.

  Yet another embodiment of the present invention is a head mounted display. The head-mounted display is a head-mounted display that can be mounted on a user's head, and a display device provided on an inner surface when the head-mounted display is mounted on the user's head, and the head-mounted display A part of the user's body that is detected by an input device that can detect that a part of the user's body is in contact with or close to the user's body is attached to the outer surface when the user is attached to the user's head. A display control unit that acquires image data generated based on input information indicating contact or proximity, and displays the acquired image data on a display device.

  Yet another embodiment of the present invention is a portable information terminal. This portable information terminal is a portable information terminal that can be carried by the user, and includes an input device that can detect that a part of the user's body is in contact with or close to the portable information terminal, and a head-mounted display that can be worn on the user's head. When the portable information terminal is mounted on the surface that is outside the head-mounted display when it is mounted on the user's head, it indicates that a part of the user's body that is detected by the input device is in contact with or in proximity A communication unit that transmits input information to a device that generates an image to be displayed on a head-mounted display based on the input information.

  Yet another embodiment of the present invention is an image processing apparatus. In this apparatus, when a head mounted display that can be mounted on the user's head is mounted on the user's head, a part of the user's body that is mounted on a surface that is outside the head mounted display is in contact with or close to the user's head. A receiving unit that receives input information detected by an input device that can detect that a part of the user's body has touched or approached, and a head-mounted display based on the input information received by the receiving unit Generating the image data to be displayed on the display device provided on the inner surface of the head mounted display when the is mounted on the user's head, and the image data generated by the generating unit A transmission unit for transmitting to the mount display.

  It should be noted that any combination of the above-described constituent elements and a representation of the present invention converted between a method, an apparatus, a system, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the convenience of the user of a head mounted display can be improved.

It is a figure which shows the use environment of the game system which concerns on embodiment. It is an external view of the head mounted display which concerns on embodiment. It is a functional lineblock diagram of a game device. It is a functional lineblock diagram of a head mount display. It is a functional block diagram of the control part of a head mounted display. It is a functional block diagram of a portable information terminal. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the touchscreen of a portable information terminal. It is a figure which shows the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the example of the image displayed on the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the touchscreen of a portable information terminal. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a figure which shows the correspondence of the display area of the display apparatus of a head mounted display, and the input area of a touch panel and a touchpad. It is a figure which shows the example of the image displayed on the display apparatus of a head mounted display. It is a sequence diagram which shows the procedure of the display control method which concerns on embodiment.

  A display system using a head mounted display (HMD) will be described. The head-mounted display is a display device that is worn on the user's head so as to cover both eyes of the user and allows viewing of still images and moving images displayed on a display screen provided in front of the user's eyes. The target displayed on the head-mounted display may be a content such as a movie or a TV program, but in this embodiment, an example in which the head-mounted display is used as a display device for displaying a game image will be described. To do.

  FIG. 1 shows a use environment of the game system 1 according to the embodiment. The game system 1 includes a game device 10 that executes a game program, an input device 20 that inputs a user instruction to the game device 10, an imaging device 14 that captures an actual space around the user, and the game device 10. And a head-mounted display 100 that displays a generated game image.

  In the present embodiment, an input device for receiving a touch input by the user is provided on a surface that is on the outer side of the user when the head mounted display 100 is mounted. When a game that can be operated only by an input to the input device is executed, the input device 20 may not be used. When the head mounted display 100 is mounted, the user cannot visually recognize the surrounding situation, and thus it is difficult to search for an input device such as the input device 20 after the head mounted display 100 is mounted. In addition, if an input device such as the input device 20 is moved greatly, it may hit an obstacle or the like around it. However, in the present embodiment, it is possible to accept input only by an input device that can accept touch input, so there is no need to search for or swing around the input device 20. Further, as will be described later, since an operation can be performed by an intuitive input interface using touch input, a user-friendly input interface that is easy to use even for a user who is not familiar with the game can be provided.

  When executing a game that accepts an operation instruction by both the input device and the input device 20, for example, as shown in FIG. 1, while holding and operating the input device 20 with the right hand, the outer surface of the head-mounted display 100 with the left hand. You may perform the input with respect to the input device provided.

  The game apparatus 10 executes the game program based on an instruction input to the input device 20 or the input device provided on the outer surface of the head mounted display 100, generates an image signal indicating the processing result of the game program, and generates the generated image The signal is transmitted to the head mounted display 100.

  The head mounted display 100 receives the image signal generated in the game apparatus 10 and displays a game image. In addition, information related to the user's touch input to the input device provided on the outer surface of the head mounted display 100 is transmitted to the game device 10. The head mounted display 100 may be connected to the game apparatus 10 by a wired cable, or may be wirelessly connected by a wireless local area network (LAN) or the like.

  The game image may be output to the display device 12. The display device 12 may be a television having a display and a speaker, or a computer display.

  The input device 20 has a function of transmitting an instruction input by a user to the game apparatus 10 and is configured as a wireless controller capable of wireless communication with the game apparatus 10 in this embodiment. The input device 20 and the game apparatus 10 may establish a wireless connection using a Bluetooth (registered trademark) (Bluetooth) protocol. The input device 20 is not limited to the wireless controller, and may be a wired controller connected to the game apparatus 10 via a cable.

  The input device 20 is driven by a battery and is configured to have a plurality of buttons for performing operation instructions for proceeding with the game. When the user operates the button of the input device 20, the operation instruction is transmitted to the game apparatus 10 by wireless communication.

  The imaging device 14 is a video camera composed of a CCD imaging device, a CMOS imaging device, or the like. The imaging device 14 images a real space at a predetermined cycle and generates a frame image for each cycle. The imaging device 14 is connected to the game device 10 via a USB (Universal Serial Bus) or other interface. The image picked up by the image pickup device 14 is used in the game device 10 to derive the positions and postures of the input device 20 and the head mounted display 100. The imaging device 14 may be a distance measuring camera or a stereo camera capable of acquiring a distance. In this case, the imaging device 14 can acquire the distance between the imaging device 14 and the input device 20 or the head mounted display 100.

  In the game system 1 of the present embodiment, the input device 20 has a light emitter configured to emit light in a plurality of colors. During the game, the illuminant emits light in the color designated by the game device 10 and is imaged by the imaging device 14. The imaging device 14 images the input device 20, generates a frame image, and supplies the frame image to the game device 10. The game apparatus 10 acquires a frame image, and derives position information of the light emitter in real space from the position and size of the image of the light emitter in the frame image. The game apparatus 10 treats the position information as a game operation instruction and reflects it in the game process such as controlling the movement of the player character. The game apparatus 10 according to the present embodiment has a function of processing a game application using not only an operation input such as a button of the input device 20 but also the positional information of the acquired light emitter image.

  The input device 20 includes an acceleration sensor and a gyro sensor. The detection value of the sensor is transmitted to the game apparatus 10 at a predetermined cycle, and the game apparatus 10 acquires the detection value of the sensor and acquires the posture information of the input device 20 in the real space. The game apparatus 10 handles the posture information as a game operation instruction and reflects it in the game processing.

  FIG. 2 is an external view of the head mounted display 100 according to the embodiment. The head mounted display 100 includes a main body part 110, a head contact part 120, a touch pad 140, and a terminal mounting part 150.

  The position information of the user can be measured by a position sensor such as GPS (Global Positioning System) incorporated in or externally attached to the head mounted display 100. Further, posture information such as the orientation and inclination of the head of the user wearing the head mounted display 100 can be measured by a posture sensor built in or externally attached to the head mounted display 100.

  The main body 110 includes a display, a position information acquisition sensor, a posture sensor, a communication device, and the like. The head contact unit 120 includes a biological information acquisition sensor capable of measuring biological information such as a user's body temperature, pulse, blood component, sweating, brain waves, and cerebral blood flow.

  An input device for receiving a touch input by a user is provided on the outer surface of the head mounted display 100. The input device may be a touch pad that is provided in advance in the head mounted display 100 and does not include a display device, or may be a touch panel that includes a display device. Further, an external device such as a smartphone provided with a touch pad or a touch panel that functions as an input device may be detachably connected to the head mounted display 100. In the present embodiment, touch pad 140 is provided on the upper surface and side surface of main body 110, and portable information terminal 200 is detachably connected to terminal mounting portion 150 provided on the front of main body 110.

  The touch pad 140 provided on the temporal region may be any type of touch pad such as a matrix switch method, a resistive film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, a capacitance method, or the like. . The touch pad 140 may be a contact-type touch pad that detects an input when a user's finger or the like touches the surface, or may detect an input even when the user's finger or the like is close to the surface. A contact type touchpad may be used. The touch pad 140 may include a pressure-sensitive sensor that can detect the pressure of pressing against the surface, or may calculate the input intensity based on the area, voltage value, capacitance, etc. of the area where the input is detected. Good. The touch pad 140 outputs the coordinates of the position where the input is detected.

  A terminal mounting portion 150 for detachably mounting the portable information terminal 200 is provided on the frontal head, and the user's portable information terminal 200 can be mounted. When the portable information terminal 200 is attached, the portable information terminal 200 functions as an input device. In addition, an image generated by the head mounted display 100 or information about the head mounted display 100 can be displayed on the display device of the portable information terminal 200.

  The head mounted display 100 may further be provided with a camera that captures the eyes of the user. The camera mounted on the head mounted display 100 can detect the user's line of sight, pupil movement, blinking, and the like.

  In the present embodiment, the head-mounted display 100 will be described. However, the display control method of the present embodiment is not limited to the head-mounted display 100 in a narrow sense, but includes glasses, glasses-type displays, glasses-type cameras, headphones, and headsets. (Headphones with microphones), earphones, earrings, ear-mounted cameras, hats, hats with cameras, hair bands, etc. can also be applied.

  FIG. 3 is a functional configuration diagram of the game apparatus 10. The game apparatus 10 includes a communication unit 302, a control unit 310, and a data holding unit 360. The control unit 310 includes an input information reception unit 311, an input position adjustment unit 312, an HMD image data generation unit 313, an HMD image data transmission unit 314, a touch panel display information generation unit 315, a touch panel display information transmission unit 316, and an HMD display information reception unit. 317, a captured image analysis unit 318, and a game control unit 319 are provided.

  The communication unit 302 controls communication with the head mounted display 100, the portable information terminal 200, the input device 20, the imaging device 14, the display device 12, and the like. The data holding unit 360 holds program data for a game executed in the game apparatus 10 and various data used by the game program.

  The input information receiving unit 311 receives, from the head mounted display 100 and the portable information terminal 200, information of touch input by the user received by the head mounted display 100 and the portable information terminal 200, respectively. The input information receiving unit 311 further receives information on the position and orientation of the head mounted display 100 from the head mounted display 100. When the input device 20 is used, the input information receiving unit 311 receives input information such as a button received by the input device 20 from the input device 20.

  The input position adjustment unit 312 associates the position of the touch input in the input information received by the input information reception unit 311 with the position of the image displayed on the head mounted display 100. In the present embodiment, since a user's touch input is received by a plurality of touch pads 140 arranged at different positions and the touch panel of portable information terminal 200, it is necessary to associate each touch area with a displayed image area. Details of the function of the input position adjustment unit 312 will be described later.

  The game control unit 319 executes the game program and advances the game based on the instruction input by the user received by the input information receiving unit 311.

  The HMD image data generation unit 313 generates a game image controlled by the game control unit 319. The HMD image data transmission unit 314 transmits the image data generated by the HMD image data generation unit 313 to the head mounted display 100.

  The touch panel display information generation unit 315 is information related to the game controlled by the game control unit 319 and generates information to be displayed on the touch panel of the portable information terminal 200. The touch panel of the portable information terminal 200 cannot be viewed by a user wearing the head mounted display 100 during a game, but can display information for other people around the user. The touch panel display information transmission unit 316 transmits the display information generated by the touch panel display information generation unit 315 to the portable information terminal 200.

  The HMD display information receiving unit 317 receives information to be displayed on the display device of the head mounted display 100 from the portable information terminal 200. The display information received by the HMD display information receiving unit 317 is included in the image displayed on the head mounted display 100 by the HMD image data generating unit 313. Thereby, even during the game, the display information transmitted from the portable information terminal 200 can be displayed on the display device of the head mounted display 100 and made visible to the user.

  The captured image analysis unit 318 acquires an image captured by the imaging device 14, analyzes the acquired image, and calculates the positions and orientations of the head mounted display 100 and the input device 20. The game control unit 319 acquires the position and orientation of the input device 20 as an instruction from the user and reflects them in the progress of the game. The HMD image data generation unit 313 controls the viewpoint position and line-of-sight direction used when generating a game image based on the position and orientation of the head mounted display 100.

  FIG. 4 is a functional configuration diagram of the head mounted display 100. The head mounted display 100 includes an input interface 22, an output interface 30, a backlight 32, a communication control unit 40, a network adapter 42, an antenna 44, a storage unit 50, a GPS unit 60, a wireless unit 62, a posture sensor 64, and an external input / output terminal. An interface 70, an external memory 72, a clock unit 80, a display device 90, a touch pad 140, and a control unit 160 are provided. These functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The control unit 160 is a main processor that processes and outputs signals such as image signals and sensor signals, commands and data. The input interface 22 receives operation signals and setting signals from the touch pad 140 and the touch pad controller and supplies them to the control unit 160. The output interface 30 receives an image signal from the control unit 160 and causes the display device 90 to display the image signal. The backlight 32 supplies a backlight to the liquid crystal display that constitutes the display device 90.

  The communication control unit 40 transmits data input from the control unit 160 to the outside through wired or wireless communication via the network adapter 42 or the antenna 44. The communication control unit 40 also receives data from the outside via wired or wireless communication via the network adapter 42 or the antenna 44 and outputs the data to the control unit 160.

  The storage unit 50 temporarily stores data, parameters, operation signals, and the like processed by the control unit 160.

  The GPS unit 60 receives position information from GPS satellites according to an operation signal from the control unit 160 and supplies the position information to the control unit 160. The radio unit 62 receives position information from the radio base station and supplies it to the control unit 160 in accordance with an operation signal from the control unit 160.

  The attitude sensor 64 detects attitude information such as the orientation and inclination of the main body 110 of the head mounted display 100. The posture sensor 64 is realized by appropriately combining a gyro sensor, an acceleration sensor, an angular acceleration sensor, and the like.

  The external input / output terminal interface 70 is an interface for connecting peripheral devices such as a USB (Universal Serial Bus) controller. The external memory 72 is an external memory such as a flash memory.

  The clock unit 80 sets time information according to a setting signal from the control unit 160 and supplies time data to the control unit 160.

  FIG. 5 is a functional configuration diagram of the control unit 160 of the head mounted display 100. The control unit 160 includes a touch input reception unit 161, an input position adjustment unit 162, an input information transmission unit 163, an HMD image data reception unit 164, and an HMD display control unit 165.

  The touch input receiving unit 161 receives an input signal from the touch pad 140. The input position adjustment unit 162 associates the input position received by the touch input reception unit 161 with the position of the area to which the input area of the touch pad 140 is assigned in the image displayed on the display device 90. The input information transmission unit 163 transmits information indicating the position of the touch input adjusted by the input position adjustment unit 162 to the game apparatus 10.

  The HMD image data receiving unit 164 receives image data to be displayed on the display device 90 from the game apparatus 10. The HMD display control unit 165 displays the image data received by the HMD image data receiving unit 164 on the display device 90.

  FIG. 6 is a functional configuration diagram of the portable information terminal 200. The portable information terminal 200 includes a button 261, a communication unit 202, a control unit 210, a data holding unit 260, and a touch panel 262. The control unit 210 includes a touch input reception unit 211, an input position adjustment unit 212, an input information transmission unit 213, a touch panel display information reception unit 214, a touch panel display control unit 215, an HMD display information generation unit 216, and an HMD display information transmission unit 217. Is provided. These functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The communication unit 202 controls communication with the game apparatus 10. The data holding unit 260 stores program data of applications executed on the portable information terminal 200, data used by the applications, and the like. The touch panel 262 displays an image generated by the control unit 210 and detects a touch input by the user.

  The control unit 210 executes a cooperation application for accepting touch input on the touch panel 262 as input for a game that is executed by the game apparatus 10 and an image is displayed on the head mounted display 100.

  Touch input reception unit 211 receives an input signal from touch panel 262. The input position adjustment unit 212 associates the input position received by the touch input reception unit 211 with the position of the area to which the input area of the touch panel 262 is assigned in the image displayed on the display device 90 of the head mounted display 100. Information regarding the area to which the input area of the touch panel 262 is assigned is acquired in advance from the head mounted display 100 or the game apparatus 10 when the cooperative application is activated. The input information transmission unit 213 transmits information indicating the position of the touch input adjusted by the input position adjustment unit 212 to the game apparatus 10.

  The touch panel display information receiving unit 214 receives information to be displayed on the touch panel 262 from the game apparatus 10. The touch panel display control unit 215 displays information received by the touch panel display information receiving unit 214 on the touch panel 262.

  The HMD display information generation unit 216 generates information to be displayed on the display device of the head mounted display 100. The HMD display information generation unit 216 acquires various types of notification information from other applications executed on the portable information terminal 200, and generates display information from the notification information according to a predetermined condition. For example, when a mail transmission / reception application of the portable information terminal 200 receives a mail, information such as a mail title, a predetermined number of character strings at the beginning of the text, and a sender may be extracted to generate display information. The HMD display information transmission unit 217 transmits the display information generated by the HMD display information generation unit 216 to the game apparatus 10. In the present embodiment, information to be displayed on the display device of the head mounted display 100 is once transmitted to the game device 10 and image data is generated by the game device 10, but in another example, the head mounted display Information to be displayed on the display device 100 may be transmitted directly to the head mounted display 100 and displayed on the display device 90 by the HMD display control unit 165 of the head mounted display 100.

  FIG. 7 shows an example of an image displayed on the display device 90 of the head mounted display 100. An image of a game controlled by the game apparatus 10 is displayed on the display device 90 of the head mounted display 100.

  FIG. 8 shows an example of an image displayed on the display device 90 of the head mounted display 100. In the game situation shown in FIG. 7, it is assumed that the surroundings suddenly turned dark and was surrounded by darkness. The HMD image data generation unit 313 of the game apparatus 10 generates a completely dark game image, and the completely dark game image is displayed on the display device 90 of the head mounted display 100.

  FIG. 9 shows the touch panel 262 of the portable information terminal 200. In the example of FIG. 9, the touch panel display information generation unit 315 of the game apparatus 10 generates information for displaying on the touch panel 262 the same image as that displayed on the display device 90 of the head mounted display 100, and the portable information terminal 200. The touch panel display control unit 215 displays the same image as the image displayed on the display device 90 of the head mounted display 100 on the touch panel 262. Here, it is assumed that there is provided a function of attaching night vision goggles by performing a downward flick input at an arbitrary position on the touch panel 262 and removing the night vision goggles by performing an upward flick input. When the user performs a downward flick input on touch panel 262, input information transmission unit 213 transmits information indicating that the downward flick input has been performed to game device 10.

  FIG. 10 shows an example of an image displayed on the display device 90 of the head mounted display 100. As shown in FIG. 9, when the user performs a flick input in the downward direction on the touch panel 262, the game control unit 319 of the game apparatus 10 indicates that the flick input has been performed in the downward direction on the touch panel 262. When the information is acquired, the HMD image data generation unit 313 is instructed to generate a night vision image, assuming that night vision goggles are attached. The HMD image data generation unit 313 generates a night vision image, and the night vision image is displayed on the display device 90 of the head mounted display 100 as shown in FIG. As described above, since the user actually moves the goggles up and down and the movement of flicking up and down on the touch panel 262 correspond to each other, an intuitive user-friendly operation system is provided. Can do.

  FIG. 11 shows an example of an image displayed on the display device 90 of the head mounted display 100. On the display device 90 of the head mounted display 100, an image of a game in which the cloudiness 402 is generated in the field of view is displayed.

  FIG. 12 shows the touch panel 262 of the portable information terminal 200. It is assumed that a function is provided to remove the cloudiness 402 displayed at the swipe position by performing swipe input in an arbitrary direction at an arbitrary position on the touch panel 262. When the user performs a swipe input on the touch panel 262, the input information transmission unit 213 transmits information indicating the swipe position to the game apparatus 10.

  FIG. 13 shows an example of an image displayed on the display device 90 of the head mounted display 100. As shown in FIG. 12, when the user performs a swipe input on the touch panel 262, the game control unit 319 of the game apparatus 10 removes the cloudiness 402 displayed at the position where the swipe input has been performed. As shown in FIG. 13, the HMD image data generation unit 313 erases the removed cloudiness 402 from the game image. In FIG. 12, the user performs a swipe input in the area on the right side of the game image displayed on the touch panel 262. However, since the user corresponds to the area on the left side of the screen when viewed from the user, the game control unit 319 The cloudiness 402 displayed in the left area of the image is erased. In addition, when the same image as the game image displayed on the head mounted display 100 is displayed on the touch panel 262, from the viewpoint of the person watching the touch panel 262, the user performs a swipe input in the area on the right side of the game image. Since the cloudiness 402 displayed in the left area of the game image is erased and gives a sense of incongruity, the touch panel 262 is a mirror image obtained by horizontally inverting the game image displayed on the head mounted display 100. Is displayed.

  FIG. 14 shows an example of an image displayed on the display device 90 of the head mounted display 100. On the display device 90 of the head mounted display 100, an image of a game in which a spider 404 is attached to the field of view is displayed.

  FIG. 15 shows the touch panel 262 of the portable information terminal 200. It is assumed that a function for removing the spider 404 displayed at the swipe position by repeatedly performing swipe input in an arbitrary direction at an arbitrary position on the touch panel 262 is provided. When the user performs a swipe input a predetermined number of times or more on touch panel 262, input information transmission unit 213 transmits information indicating the swipe position and the number of times to game device 10. In another example, the input information transmission unit 213 transmits information indicating the swipe position to the game device 10 every time a swipe input is received, and the game control unit 319 of the game device 10 causes the user to perform a swipe input a predetermined number of times or more. It may be determined whether or not.

  FIG. 16 shows an example of an image displayed on the display device 90 of the head mounted display 100. As shown in FIG. 15, when the user performs a swipe input on the touch panel 262 a predetermined number of times or more, the game control unit 319 of the game apparatus 10 is displayed at a position where the swipe input is performed a predetermined number of times or more. The spider 404 is removed. As shown in FIG. 16, the HMD image data generation unit 313 erases the removed spider 404 from the game image.

  FIG. 17 shows an example of an image displayed on the display device 90 of the head mounted display 100. On the display device 90 of the head mounted display 100, an image of a game in which a flash van (flash acoustic grenade) 406 is arranged in the game field is displayed.

  FIG. 18 shows an example of an image displayed on the display device 90 of the head mounted display 100. In the game situation shown in FIG. 17, when a predetermined condition for the flash van 406 to explode is satisfied, the game control unit 319 causes the flash van 406 to explode. The HMD image data generation unit 313 generates a game image in which flash is generated by the explosion of the flash van 406, and the flash 408 is displayed on the display device 90 of the head mounted display 100.

  Here, when the flash 408 is generated, if the user touches the touch panel 262 so that both eyes are covered with hands, the surrounding game field is displayed again as the flash 408 disappears. If the touch panel 262 is not touched so as to cover the screen, the flash 408 is blinded, and even if the flash 408 disappears, a white game image is displayed for a predetermined time, and the surrounding game field is not displayed.

  FIG. 19 shows the touch panel 262 of the portable information terminal 200. In each of the right-half area and the left-half area of the touch panel 262, a function is provided in which the palm is brought into contact with the touch panel 262 to cover both eyes and protect from the flash 408. When touch input is performed on the touch panel 262, the input information transmission unit 213 transmits information indicating a plurality of touched positions to the game apparatus 10. The game control unit 319 determines whether or not touch input corresponding to a predetermined condition indicating that the palm is touched is performed in each of the right half area and the left half area of the touch panel 262 at the moment of the explosion. If it has been determined and has been performed, the HMD image data generation unit 313 causes the surrounding game field to be displayed again with the disappearance of the flash 408. If it has not been performed, the predetermined time remains even if the flash 408 disappears. A white game image is displayed.

  FIG. 20 shows the touch panel 262 of the portable information terminal 200. As shown in FIG. 20, when a user makes a peace sign with his / her finger and touches the touch panel 262 with the index and middle fingers in the vicinity of the eyes of either of the left and right eyes, a virtual player in a player character or first-person viewpoint game operated by the user Assume that a beam is fired from the eyes of a typical user.

  FIG. 21 shows an example of an image displayed on the display device 90 of the head mounted display 100. A function of emitting a beam is provided by performing touch input simultaneously at two points within an area of a predetermined range of the touch panel 262. When touch input is performed on the touch panel 262, the input information transmission unit 213 transmits information indicating a plurality of touched positions to the game apparatus 10. The game control unit 319 determines whether or not two touch inputs are simultaneously performed within a predetermined range, and if so, the HMD image data generation unit 313 indicates that the beam 410 is emitted. If not displayed, the beam 410 is not fired. When the beam 410 is fired, the game control unit 319 calculates a path of the beam 410, and when an object exists on the path, the game control unit 319 executes processing when the beam 410 hits the object.

  FIG. 22 shows the touch panel 262 of the portable information terminal 200. As shown in FIG. 22, when the user performs touch input with a finger at an arbitrary position on the touch panel 262 for a predetermined time or longer, the finger is placed at a corresponding position on the game image displayed on the display device 90 of the head mounted display 100. Assume that an image is displayed.

  FIG. 23 shows an example of an image displayed on the display device 90 of the head mounted display 100. By performing touch input at an arbitrary position on the touch panel 262 with a pressure of a predetermined time or more or a predetermined value or more, the user's finger enters the virtual world image displayed on the display device 90 of the head mounted display 100 from the back. A function for displaying such a video effect is provided. When touch input is performed on the touch panel 262, the input information transmission unit 213 transmits information indicating a plurality of touched positions and time or pressure to the game apparatus 10. The game control unit 319 determines whether or not touch input has been performed with a pressure equal to or longer than a predetermined time or a predetermined value, and if so, the position at which the touch input has been performed on the HMD image data generation unit 313. If the user's finger 412 is not displayed, the finger 412 is not displayed. It may be displayed that the longer the time that touch input has been performed or the greater the pressure, the more the finger enters the near side.

  As described above, when an input corresponding to an action performed by the user using the hand near the head in the real world is performed on the touch panel 262, a result to be generated in the real world is reflected on the display by the action. It is possible to provide a user interface that is intuitive and easy to understand.

  In the above example, the same image as that displayed on the display device 90 of the head mounted display 100 is displayed on the touch panel 262 of the portable information terminal 200. Thereby, the surrounding people can also enjoy watching the game that the user is running. For example, this is also effective when the parent wants to know the game being executed by the child. Moreover, since the surrounding people can see the game image without displaying the game image on the display device 12, the display device 12 can be used for other purposes.

  An image different from the image displayed on the display device 90 may be displayed on the touch panel 262. The content displayed on the touch panel 262 may be determined by receiving an instruction from the user via the head mounted display 100, the portable information terminal 200, or the game apparatus 10, or the head mounted display 100, the portable information terminal 200, Alternatively, it may be automatically determined by the game apparatus 10.

  FIG. 24 shows an example of an image displayed on the touch panel 262 of the portable information terminal 200. In the example of FIG. 24, information indicating the status of the game controlled by the game control unit 319 is displayed on the touch panel 262. The touch panel display information generation unit 315 of the game apparatus 10 acquires information indicating the state of the game controlled by the game control unit 319 and generates display information. The touch panel 262 displays that the game status is “matching with boss”, the remaining physical strength value of the enemy character indicating the game status in more detail, and a message from the user to the surrounding people Yes.

  FIG. 25 shows an example of an image displayed on the touch panel 262 of the portable information terminal 200. Also in the example of FIG. 25, information indicating the state of the game controlled by the game control unit 319 is displayed on the touch panel 262. The touch panel display information generation unit 315 of the game apparatus 10 acquires information indicating the state of the game controlled by the game control unit 319 and generates display information. The touch panel 262 displays that the game status is “watching movie”, the remaining number of seconds of the movie scene showing the game status in more detail, and a message from the user to surrounding people. .

  FIG. 26 shows an example of an image displayed on the touch panel 262 of the portable information terminal 200. Also in the example of FIG. 25, information indicating the state of the game controlled by the game control unit 319 is displayed on the touch panel 262. The touch panel display information generation unit 315 of the game apparatus 10 acquires information indicating the state of the game controlled by the game control unit 319 and generates display information. The touch panel 262 displays a game play time and a rough estimate of the time required until the next save point. For example, this is also effective when the parent wants to limit the play time of the child game.

  In this manner, by displaying information indicating the situation such as a game being executed by the user of the head mounted display 100 or a moving image being viewed on the touch panel 262, the surrounding people can grasp the user's situation. Since it can, when talking to the user, it is possible to choose and talk to a situation where there is no problem. Therefore, the situation where the user is interrupted in a situation where the user does not want to be interrupted can be reduced, and the user can obtain an immersive feeling more effectively.

  FIG. 27 shows an example of an image displayed on the touch panel 262 of the portable information terminal 200. In the example of FIG. 27, an image serving as a marker for the captured image analysis unit 318 to analyze the position and orientation of the head mounted display 100 is displayed on the touch panel 262. Thereby, the precision at the time of analyzing the position and attitude | position of the head mounted display 100 can be improved. It is more effective to display different markers on a plurality of touch panels provided at a plurality of positions. Note that the image to be a marker displayed on the touch panel 262 may be an arbitrary figure such as a barcode or an image such as a character. Further, a portion around the eyes of a face such as a person or character may be displayed so that the user is wearing the character's face. Thereby, surrounding people can also be entertained.

  FIG. 28 shows an example of an image displayed on the display device 90 of the head mounted display 100. When a mail application of the portable information terminal 200 receives a mail during the game, the HMD display information generation unit 216 acquires information about the received mail and displays information to be displayed on the display device 90 of the head mounted display 100. Generate. When the HMD display information receiving unit 317 of the game apparatus 10 receives information to be displayed on the display device 90 of the head mounted display 100 from the portable information terminal 200, the HMD image data generating unit 313 receives image data including the received information. Generate. In the example of FIG. 28, the sender of the mail received by the portable information terminal 200, the title, and the content at the beginning of the text are displayed in the display area 502 at the bottom of the screen on the display device 90 of the head mounted display 100. .

  The setting of information to be displayed on the display device 90 of the head mounted display 100 may be received from the user in advance via the touch panel 262 of the portable information terminal 200. As a result, the user can view necessary information regarding the portable information terminal 200 even while the head mounted display 100 is being used. If necessary, the user can use the portable information terminal 200 by interrupting viewing of the head mounted display 100. be able to. At this time, the information displayed on the display device 90 of the head mounted display 100 may not be displayed on the touch panel 262 of the portable information terminal 200. Thereby, it can prevent that a user's information is seen by the surrounding people.

  The display area 502 for displaying information received from the portable information terminal 200 may be determined in position, size, etc. depending on the game situation, or may receive settings from the user. Since the aspect ratio of the image to be displayed and the aspect ratio of the display area of the display device 90 do not match, if there are areas where no image is displayed on the top and bottom or the left and right of the screen, that area can be used as the display area 502. Good.

  The portable information terminal 200 may be operable while viewing the display device 90 of the head mounted display 100. When performing an operation using the input function of the game device 10 such as the input device 20, the game device 10 transmits the instruction input received from the input device 20 or the like to the portable information terminal 200, and the cooperative application of the portable information terminal 200 The received instruction input is transmitted to an application to be operated, for example, a mail application. When an operation is performed using the input function of the head mounted display 100 such as the touch pad 140, the head mounted display 100 receives the instruction input received from the touch pad 140 or the like directly or via the game device 10. Send to. When an operation is performed using the touch panel 262 of the portable information terminal 200, an application to be operated receives an instruction input directly or via a cooperative application.

  FIG. 29 shows a correspondence relationship between the display area of the display device 90 of the head mounted display 100 and the input areas of the touch panel 262 and the touch pad 140. As described above, in the present embodiment, since the plurality of touch panels 262 and the touch pad 140 are provided, the input position adjustment unit 312 of the game apparatus 10 determines the positions of the inputs to the touch panel 262 and the touch pad 140. Then, it is associated with the position in the image displayed on the display device 90 of the head mounted display 100. In the example of FIG. 29, among the images displayed on the display device 90 of the head mounted display 100, the area a is the touch panel 262 of the portable information terminal 200, and the area b is the touch pad 140 provided on the upper surface of the head mounted display 100. The area c is associated with the touch pad 140 provided on the left head of the head mounted display 100, and the area d is associated with the touch pad 140 provided on the right head of the head mounted display 100. At this time, it is desirable that the user's field of view and the actual installation positions of the touch panel 262 and the touch pad 140 be as similar as possible. Thereby, it is possible to provide a user interface that allows the user to operate without feeling a contradiction between the body sensation and vision.

  When the display area and the input area of the touch panel 262 or the touch pad 140 are different in size or shape, the input position adjustment unit 212 of the portable information terminal 200 is used for the touch panel 262, and the head mounted display is used for the touch pad 140. Each of the 100 input position adjustment units 162 associates the actual input position with the input position in the assigned input area.

  FIG. 30 shows an example of an image displayed on the display device 90 of the head mounted display 100. When the input information receiving unit 311 receives the input information, the input position adjusting unit 312 of the game apparatus 10 displays a marker 504 indicating the input position at the adjusted position, and on the touch panel 262 or the touch pad 140 that has received the input. A corresponding area frame 506 is displayed. Thus, even when a plurality of touch panels 262 and touch pads 140 are provided, it is easy to understand which touch panel 262 or touch pad 140 should be touched for the user to input a desired position. be able to. Further, it is possible to easily indicate which part of the touch panel 262 or the touch pad 140 should be touched.

  FIG. 31 is a sequence diagram illustrating a procedure of the display control method according to the embodiment. The game control unit 319 of the game apparatus 10 controls the game (S100), and the HMD image data generation unit 313 generates image data to be displayed on the head mounted display 100 (S102). The HMD image data transmission unit 314 transmits the image data to the head mounted display 100 (S104), and the HMD display control unit 165 of the head mounted display 100 displays the image data on the display device 90 (S106).

  The touch panel display information generation unit 315 of the game apparatus 10 generates information for displaying the game status on the touch panel 262 of the portable information terminal 200 (S108), and the touch panel display information transmission unit 316 displays the display information on the portable information terminal. 200 (S110). The touch panel display control unit 215 of the portable information terminal 200 displays the display information on the touch panel 262 (S112).

  When the user performs touch input on the touch panel 262 of the portable information terminal 200, the touch input reception unit 211 receives input (S120), the input position adjustment unit 212 adjusts the position of the touch input, and the input information transmission unit 213. Transmits the input information to the game apparatus 10 (S124). When the user performs a touch input on the touch pad 140 of the head mounted display 100, the touch input receiving unit 161 receives an input (S122), the input position adjusting unit 162 adjusts the position of the touch input, and the input information transmitting unit. 163 transmits the input information to the game apparatus 10 (S126). When the input information receiving unit 311 of the game apparatus 10 receives input information from the portable information terminal 200 or the head mounted display 100, the input position adjusting unit 312 adjusts the position of the touch input, and the game control unit 319 controls the touch input. (S128) and return to S100.

  When the HMD display information generation unit 216 generates information to be displayed on the head mounted display 100 by receiving the e-mail or the like (S140), the HMD display information transmission unit 217 sends the display information to the game device 10. Transmit (S142). When the HMD display information receiving unit 317 of the game apparatus 10 receives display information from the portable information terminal 200, the HMD image data generating unit 313 reflects the display information on the display (S144), and the process returns to S102.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to each component and combination of processing processes, and such modifications are within the scope of the present invention.

  In the above example, a monocular image is displayed on the display device 90 of the head mounted display 100. However, in another example, a binocular stereoscopic image may be displayed. In this case, a binocular stereoscopic image may be displayed on the touch panel 262 of the portable information terminal 200, or a monocular image may be displayed on the touch panel 262.

  In the above example, the game apparatus 10 controls the game and generates image data to be displayed on the display device 90 of the head mounted display 100. However, in another example, the configuration for controlling the game and generating image data. May be provided in the head mounted display 100. In this case, the game apparatus 10 may not be provided, and the portable information terminal 200 and the head mounted display 100 may communicate directly without using the game apparatus 10.

  In the above example, the head mounted display 100 is used in the game system. However, the technology described in the embodiment can also be used when displaying contents other than games. Even in a display system that does not have a dedicated input device such as a game controller, according to the head mounted display 100 according to the embodiment, a user instruction can be easily input via a touch pad or a touch panel of a portable information terminal. be able to.

  10 game devices, 90 display devices, 100 head-mounted displays, 140 touch pads, 161 touch input reception units, 162 input position adjustment units, 163 input information transmission units, 164 HMD image data reception units, 165 HMD display control units, 200 mobile phones Information terminal, 211 Touch input reception unit, 212 Input position adjustment unit, 213 Input information transmission unit, 214 Touch panel display information reception unit, 215 Touch panel display control unit, 216 HMD display information generation unit, 217 HMD display information transmission unit, 262 Touch panel 311 Input information receiving unit, 312 Input position adjusting unit, 313 HMD image data generating unit, 314 HMD image data transmitting unit, 315 Touch panel display information generating unit, 316 Touch panel display information transmitting unit, 317 HMD display information receiving unit 319 game control unit.

Claims (9)

  1. A head-mounted display system having a head-mounted display that can be worn on a user's head,
    A display device provided on the inner surface when the head mounted display is mounted on the user's head; and
    A communication unit that accepts an operation input from an input device held by a user;
    A display control unit for acquiring image data generated based on a user operation input, and displaying the acquired image data on the display device;
    A head-mounted display system comprising:
  2. An input device that is provided on the outer surface when the head-mounted display is mounted on the user's head, and that can detect that a part of the user's body is in contact with or close to the head;
    A main body to which a portable information terminal is detachably connected;
    The head mounted display system according to claim 1, further comprising:
  3. A first input unit provided in the main body unit for receiving an operation from a user;
    The head-mounted display system according to claim 2, wherein the first input unit includes at least one of a button and a sensor for receiving an operation from a user.
  4. The second input unit further comprising a second input unit different from the first input unit, wherein the second input unit includes at least one of a button and a sensor for receiving an operation from a user. The head mounted display system described in 1.
  5.   The head mounted display system according to claim 4, wherein the second input unit is mounted on the input device.
  6.   6. The head mounted display system according to claim 4, wherein the function allocated to the first input unit includes a function similar to the function allocated to the second input unit.
  7. A head mounted display that can be worn on the user's head,
    A display device provided on the inner surface when the head mounted display is mounted on the user's head; and
    A display control unit that acquires image data generated based on an operation input from an input device held by a user, and displays the acquired image data on the display device;
    A head-mounted display comprising:
  8. Computer
    An acquisition unit that acquires data of an image generated based on an operation input from an input device held by a user wearing a head-mounted display that can be mounted on the user's head;
    A display control unit for displaying image data acquired by the acquisition unit on a display device provided on an inner surface when the head mounted display is mounted on a user's head;
    Display control program to function as.
  9. An acquisition unit acquiring image data generated based on an operation input from an input device held by a user wearing a head mounted display that can be mounted on the user's head;
    The display control unit displaying the image data acquired by the acquisition unit on a display device provided on an inner surface when the head mounted display is mounted on a user's head; and
    A display control method comprising:
JP2018092931A 2018-05-14 2018-05-14 Head-mount display system, head-mount display, display control program, and display control method Pending JP2018160249A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2018092931A JP2018160249A (en) 2018-05-14 2018-05-14 Head-mount display system, head-mount display, display control program, and display control method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2018092931A JP2018160249A (en) 2018-05-14 2018-05-14 Head-mount display system, head-mount display, display control program, and display control method

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
JP2014215565 Division 2014-10-22

Publications (1)

Publication Number Publication Date
JP2018160249A true JP2018160249A (en) 2018-10-11

Family

ID=63795715

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2018092931A Pending JP2018160249A (en) 2018-05-14 2018-05-14 Head-mount display system, head-mount display, display control program, and display control method

Country Status (1)

Country Link
JP (1) JP2018160249A (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003289484A (en) * 2002-03-28 2003-10-10 Sanyo Electric Co Ltd Eye mount video display apparatus and operating apparatus
JP2004085476A (en) * 2002-08-28 2004-03-18 Sony Corp Head tracking method and device
WO2005022236A1 (en) * 2003-09-03 2005-03-10 Human Sense Tech Co., Ltd. Hmd apparatus having a memory slot built-in thereof
JP2008028552A (en) * 2006-07-19 2008-02-07 Nikon Corp Display device
US20100079356A1 (en) * 2008-09-30 2010-04-01 Apple Inc. Head-mounted display apparatus for retaining a portable electronic device with display
WO2013088725A1 (en) * 2011-12-16 2013-06-20 Sony Corporation Head-mounted display and information display apparatus
US20130181888A1 (en) * 2012-01-18 2013-07-18 Sony Corporation Head-mounted display
JP2013190925A (en) * 2012-03-13 2013-09-26 Nikon Corp Input device, and a display device
EP2674850A2 (en) * 2012-06-13 2013-12-18 Sony Corporation Head-mounted display

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003289484A (en) * 2002-03-28 2003-10-10 Sanyo Electric Co Ltd Eye mount video display apparatus and operating apparatus
JP2004085476A (en) * 2002-08-28 2004-03-18 Sony Corp Head tracking method and device
US20050256675A1 (en) * 2002-08-28 2005-11-17 Sony Corporation Method and device for head tracking
WO2005022236A1 (en) * 2003-09-03 2005-03-10 Human Sense Tech Co., Ltd. Hmd apparatus having a memory slot built-in thereof
JP2008028552A (en) * 2006-07-19 2008-02-07 Nikon Corp Display device
US20100079356A1 (en) * 2008-09-30 2010-04-01 Apple Inc. Head-mounted display apparatus for retaining a portable electronic device with display
JP2013125247A (en) * 2011-12-16 2013-06-24 Sony Corp Head-mounted display and information display apparatus
WO2013088725A1 (en) * 2011-12-16 2013-06-20 Sony Corporation Head-mounted display and information display apparatus
US20130181888A1 (en) * 2012-01-18 2013-07-18 Sony Corporation Head-mounted display
JP2013150118A (en) * 2012-01-18 2013-08-01 Sony Corp Head-mounted display
JP2013190925A (en) * 2012-03-13 2013-09-26 Nikon Corp Input device, and a display device
EP2674850A2 (en) * 2012-06-13 2013-12-18 Sony Corporation Head-mounted display
JP2013258573A (en) * 2012-06-13 2013-12-26 Sony Corp Head-mounted video display device

Similar Documents

Publication Publication Date Title
US8139087B2 (en) Image presentation system, image presentation method, program for causing computer to execute the method, and storage medium storing the program
US9690376B2 (en) Wireless wrist computing and control device and method for 3D imaging, mapping, networking and interfacing
JP6393367B2 (en) Tracking display system, tracking display program, tracking display method, wearable device using them, tracking display program for wearable device, and operation method of wearable device
EP2650755A2 (en) Content transfer via skin input
US20050059489A1 (en) Motion sensing applications
CN103480154B (en) Obstacle circumvention devices and methods to circumvent the obstacle
CN104076513B (en) The head-mounted display apparatus, a control method of the apparatus, and a display head mounted display system
WO2014085789A1 (en) Direct hologram manipulation using imu
JP2018508805A (en) Method and system for user interaction with the virtual or extended in reality scene using a head-mounted display
US9462633B2 (en) Mobile terminal operation controlled by proximity sensors and heart rate monitor
CN103500061B (en) A display control method and apparatus
US9884248B2 (en) Display control method for head-mounted display (HMD) and image generation device
CN103426282A (en) And a terminal remote control method
JPH0749744A (en) Head mounting type display input device
JP2013258614A (en) Image generation device and image generation method
US20150323993A1 (en) Systems and methods for providing haptic feedback for remote interactions
US20120135803A1 (en) Game device utilizing stereoscopic display, method of providing game, recording medium storing game program, and game system
KR20140070326A (en) Mobile device providing 3d interface and guesture controlling method thereof
US10269180B2 (en) Information processing apparatus and information processing method, display apparatus and display method, and information processing system
EP3098689B1 (en) Image display device and image display method
JP2011108256A (en) Information processing program, information processing method, information processing apparatus, and information processing system
US20130241927A1 (en) Computer device in form of wearable glasses and user interface thereof
US9729864B2 (en) Camera based safety mechanisms for users of head mounted displays
WO2015180497A1 (en) Motion collection and feedback method and system based on stereoscopic vision
CN105319714A (en) Display apparatus, method for controlling display apparatus, and program

Legal Events

Date Code Title Description
A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20190131

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20190226

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20190408