WO2019039378A1 - 情報処理装置および画像表示方法 - Google Patents

情報処理装置および画像表示方法 Download PDF

Info

Publication number
WO2019039378A1
WO2019039378A1 PCT/JP2018/030421 JP2018030421W WO2019039378A1 WO 2019039378 A1 WO2019039378 A1 WO 2019039378A1 JP 2018030421 W JP2018030421 W JP 2018030421W WO 2019039378 A1 WO2019039378 A1 WO 2019039378A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
display
user
head
hmd
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2018/030421
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
征志 中田
泰史 奥村
ゆりか 村瀬
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Interactive Entertainment Inc
Original Assignee
Sony Interactive Entertainment Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Interactive Entertainment Inc filed Critical Sony Interactive Entertainment Inc
Priority to US16/631,568 priority Critical patent/US11231587B2/en
Publication of WO2019039378A1 publication Critical patent/WO2019039378A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/211—Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B27/0172—Head mounted characterised by optical features
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/212—Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25—Output arrangements for video game devices
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
    • A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
    • A63F13/428—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
    • A63F13/525—Changing parameters of virtual cameras
    • A63F13/5255—Changing parameters of virtual cameras according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/55—Controlling game characters or game objects based on the game progress
    • A63F13/57—Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game
    • A63F13/573—Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game using trajectories of game objects, e.g. of a golf ball according to the point of impact
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/60—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
    • A63F13/61—Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor using advertising information
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/163—Wearable computers, e.g. on a belt
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09F—DISPLAYING; ADVERTISING; SIGNS; LABELS OR NAME-PLATES; SEALS
    • G09F19/00—Advertising or display means not otherwise provided for
    • G09F19/12—Advertising or display means not otherwise provided for using special optical effects
    • G09F19/125—Stereoscopic displays; 3D displays
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2250/00—Miscellaneous game characteristics
    • A63F2250/22—Miscellaneous game characteristics with advertising
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8082—Virtual reality
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user

Definitions

  • the present invention relates to a technique for displaying an image on a head mounted display.
  • a head mounted display is mounted on the user's head to provide the user with a virtual reality (VR) or augmented reality (AR) visual world.
  • VR virtual reality
  • AR augmented reality
  • the HMD is connected to a game device, and a user operates a game controller to play a game while watching a game image displayed on the HMD. Since the HMD provides a VR image to the entire field of view of the user, the HMD has the effect of enhancing the user's sense of immersion into the video world and dramatically improving the entertainment properties of the game and the like.
  • By providing the head tracking function to the HMD and generating an image of a virtual three-dimensional space in conjunction with the posture of the head of the user it is possible to further enhance the sense of immersion in the video world.
  • the present invention has been made in view of these problems, and one object is to provide a technique for effectively presenting additional content such as an advertisement to a user in a VR space or an AR space.
  • an information processing apparatus includes a direction detection unit that detects a gaze direction of a user wearing a head mount display or a head direction, and a display target on the head mount display In order to promote recognition of a predetermined thing or service that is different from the main content, together with the image generation unit that generates the first image related to the main content selected by the user, and the first image generated by the image generation unit And a display control unit that causes the head mounted display to display a second image related to the content of The display control unit controls the display mode of the second image on the head mounted display according to the user's gaze direction or head direction detected by the direction detection unit.
  • Another aspect of the present invention is an image display method.
  • the method comprises the steps of: detecting a gaze direction or head direction of a user wearing the head mounted display; generating a first image related to main content selected by the user as a display target on the head mounted display; Displaying on the head mounted display a second image regarding the content different from the main content and promoting the recognition of a predetermined thing or service together with the one image, and the direction of the user's gaze or head orientation And controlling the display mode of the second image on the head mounted display.
  • FIG. 5A and FIG. 5B schematically show images displayed on the HMD.
  • FIG. 6A and FIG. 6B schematically show images displayed on the HMD.
  • 7 (a) and 7 (b) are diagrams schematically showing images displayed on the HMD.
  • FIGS. 8A and 8B schematically show images displayed on the HMD.
  • FIG. 9A and FIG. 9B schematically show images displayed on the HMD.
  • FIGS. 10A and 10B schematically show images displayed on the HMD.
  • the information processing apparatus causes the head mounted display (HMD) worn by the user to display an image related to the main content selected by the user.
  • the main content is content to be mainly displayed in the HMD, and includes various video content that can be viewed by the user based on data distributed live from an external device or data recorded in advance on the information processing device.
  • the main content includes entertainment content such as a game, a concert, and a talk program.
  • a game is illustrated as the main content.
  • the information processing apparatus is an image different from the main content, and an image related to a content (hereinafter, also referred to as “additional content”) for promoting recognition of a predetermined thing or service together with the main content image.
  • the additional content includes advertisements of various things or services, and also includes content (for example, a link image to a game or a concert) for guiding the user to a game or a concert different from the main content.
  • the data of the additional content may be distributed live from an external device or may be recorded in advance in the information processing device.
  • the information processing apparatus controls the display mode of the image of the additional content in the HMD in accordance with the direction of the line of sight of the user wearing the HMD or the direction of the head. For example, the display mode of the image of the additional content is changed from the first mode to the second mode. Thereby, additional content can be effectively presented to the user in the VR space and in the AR space.
  • an advertisement is illustrated as the additional content.
  • the VR image is displayed on the non-transmissive HMD in the embodiment
  • the display technology of the additional content image in the embodiment can be applied to the case where an AR image is displayed on the optical transmissive HMD.
  • FIG. 1 shows a configuration example of an information processing system 1 in the embodiment.
  • the information processing system 1 includes an information processing apparatus 10, an HMD 100, an input device 16 operated by a user with a finger, an imaging device 14 imaging a user wearing the HMD 100, and an output device 15 displaying an image.
  • the output device 15 may be a television.
  • the information processing apparatus 10 is connected to an external network such as the Internet via an access point (AP) 17.
  • the AP 17 has the functions of a wireless access point and a router, and the information processing apparatus 10 may be connected to the AP 17 by a cable or may be connected by a known wireless communication protocol.
  • the information processing apparatus 10 of the embodiment is connected to the server 3 on the external network via the AP 17.
  • the server 3 stores data of a plurality of types of advertisements to be displayed on the HMD 100, and transmits data of the plurality of types of advertisements to the information processing apparatus 10.
  • the data of the advertisement may include an advertisement image, and may include data specifying the display timing and display mode of the advertisement.
  • the HMD 100 is mounted on the head of the user to provide the virtual reality (VR) image world to the user.
  • VR virtual reality
  • the information processing device 10 includes a processing device 11, an output control device 12, and a storage device 13.
  • the processing device 11 is a terminal device that receives operation information input to the input device 16 by the user and executes an application such as a game.
  • the processing device 11 and the input device 16 may be connected by a cable and may be connected by a known wireless communication protocol.
  • the processing apparatus 11 of the embodiment has a function of receiving the attitude information of the HMD 100 as operation information of the user with respect to the game and advancing the game.
  • the output control unit 12 is a processing unit that outputs the image data generated by the processing unit 11 to the HMD 100.
  • the output control unit 12 and the HMD 100 may be connected by a cable, and are connected by a known wireless communication protocol. It is also good.
  • the imaging device 14 is a stereo camera, and captures a user wearing the HMD 100 at a predetermined cycle, and supplies a captured image to the processing device 11.
  • the HMD 100 is provided with a marker (tracking LED) for tracking the user's head, and the processing device 11 detects the movement of the HMD 100 based on the position of the marker included in the captured image.
  • a posture sensor acceleration sensor and gyro sensor
  • the processing device 11 obtains sensor data detected by the posture sensor from the HMD 100, thereby achieving high accuracy along with the use of the photographed image of the marker.
  • the output device 15 is not necessarily required for the user wearing the HMD 100, but another user can view the display image of the output device 15 by preparing the output device 15.
  • the output control device 12 or the processing device 11 may cause the output device 15 to display the same image as the image viewed by the user wearing the HMD 100, but may display another image.
  • the output device 15 may display a game image from the viewpoint of the character of the other user.
  • the HMD 100 is a display device that displays an image on a display panel located in front of the eyes when the user wears the head.
  • the HMD 100 separately displays an image for the left eye on the display panel for the left eye and an image for the right eye on the display panel for the right eye. These images constitute parallax images viewed from the left and right viewpoints to realize stereoscopic vision. Since the user views the display panel through the optical lens, the information processing apparatus 10 supplies the HMD 100 with parallax image data in which the optical distortion due to the lens has been corrected.
  • the optical distortion correction process may be performed by either the processing device 11 or the output control device 12.
  • the processing device 11, the storage device 13, the output device 15, the input device 16, and the imaging device 14 may construct a conventional game system.
  • the processing device 11 is a game device that executes a game
  • the input device 16 is a device such as a game controller, a keyboard, a mouse, and a joystick that supplies operation information by the user to the processing device 11.
  • the storage device 13 stores system software, game software, and the like.
  • the function of the output control device 12 may be incorporated into the processing device 11. That is, the processing unit of the information processing device 10 may be configured of one processing device 11 or may be configured of the processing device 11 and the output control device 12. In the following, the function of providing the VR image to the HMD 100 will be collectively described as the function of the information processing apparatus 10.
  • the user character is moved in the virtual three-dimensional space based on the operation information input from the input device 16, and the gaze direction of the user character is determined based on the posture information of the HMD 100 mounted on the user's head. Determined.
  • the game uses the attitude information of the HMD 100 as operation information for controlling the line-of-sight direction of the user character, but the input information in the input device 16 may be used as operation information for controlling the line-of-sight direction. Good.
  • the operation information for moving the user character may use the position information of the HMD 100 without using the input information of the input device 16.
  • the operation information of the user character may be obtained from any of the input device 16 and the HMD 100, or may be obtained from another user interface.
  • the information processing apparatus 10 detects the position coordinates and orientation of the user's head (in fact, the HMD 100) by performing head tracking processing of the user.
  • the position coordinates of the HMD 100 are position coordinates in a three-dimensional space with the reference position as the origin, and the reference position may be position coordinates (latitude, longitude) when the power of the HMD 100 is turned on.
  • the attitude of the HMD 100 is an inclination in the direction of three axes with respect to the reference attitude in the three-dimensional space.
  • the reference posture may be a posture in which the user's gaze direction is horizontal, and may be set when the power of the HMD 100 is turned on.
  • the information processing apparatus 10 can detect the position coordinates and attitude of the HMD 100 only from the sensor data detected by the attitude sensor of the HMD 100, and further performs image analysis on the markers (LEDs for tracking) of the HMD 100 photographed by the imaging apparatus 14
  • the position coordinates and attitude of the HMD 100 can be detected with high accuracy.
  • the information processing apparatus 10 calculates the position of the user character in the virtual three-dimensional space based on the position information of the HMD 100, and calculates the gaze direction of the user character based on the posture information of the HMD 100.
  • the position of the user character in the virtual three-dimensional space is calculated based on the operation information of the input device 16.
  • FIG. 2 shows an example of the appearance of the HMD 100.
  • the HMD 100 includes an output mechanism unit 102 and a mounting mechanism unit 104.
  • the mounting mechanism unit 104 includes a mounting band 106 for fixing the HMD 100 to the head by circling the head by being worn by the user.
  • the mounting band 106 is made of a material or structure that can be adjusted in length in accordance with the head circumference of the user.
  • the output mechanism unit 102 includes a housing 108 shaped to cover the eyes of the left and right eyes in a state where the user wears the HMD 100, and internally includes a display panel that faces the eyes when worn.
  • the display panel may be a liquid crystal panel or an organic EL panel.
  • the housing 108 is further provided with a pair of left and right optical lenses located between the display panel and the eyes of the user and enlarging the viewing angle of the user.
  • the HMD 100 may further include a speaker and an earphone at a position corresponding to the user's ear, and may be configured to be connected with an external headphone.
  • Luminescent markers 110 a, 110 b, 110 c, and 110 d are provided on the outer surface of the housing 108.
  • the tracking LED constitutes the light emission marker 110, but it may be another type of marker, and in any case, it can be imaged by the imaging device 14 and the information processing device 10 can analyze the marker position. I hope there is.
  • the number and arrangement of the light emission markers 110 are not particularly limited, but they need to be the number and arrangement for detecting the attitude of the HMD 100, and are provided at the four corners of the front surface of the housing 108 in the illustrated example.
  • the light emission marker 110 may be provided on the side or the rear of the mounting band 106 so that the user can shoot even when the user turns his back to the imaging device 14.
  • the HMD 100 may be connected to the information processing apparatus 10 by a cable or may be connected by a known wireless communication protocol.
  • the HMD 100 transmits sensor data detected by the posture sensor to the information processing apparatus 10, receives image data generated by the information processing apparatus 10, and displays the image data on the left-eye display panel and the right-eye display panel.
  • FIG. 3 shows functional blocks of the HMD 100.
  • the control unit 120 is a main processor that processes and outputs various data such as image data, voice data, sensor data, and instructions.
  • the storage unit 122 temporarily stores data and instructions processed by the control unit 120.
  • the attitude sensor 124 detects attitude information of the HMD 100.
  • the attitude sensor 124 includes at least a 3-axis acceleration sensor and a 3-axis gyro sensor.
  • the communication control unit 128 transmits data output from the control unit 120 to the external information processing apparatus 10 by wired or wireless communication via a network adapter or an antenna.
  • the communication control unit 128 also receives data from the information processing apparatus 10 by wired or wireless communication via a network adapter or an antenna, and outputs the data to the control unit 120.
  • control unit 120 When the control unit 120 receives image data and audio data from the information processing apparatus 10, the control unit 120 supplies the image data and the audio data to the display panel 130 for display and also supplies the audio output unit 132 for audio output.
  • the display panel 130 includes a display panel 130a for the left eye and a display panel 130b for the right eye, and a pair of parallax images are displayed on each display panel. Further, the control unit 120 causes the communication control unit 128 to transmit the sensor data from the posture sensor 124 and the voice data from the microphone 126 to the information processing apparatus 10.
  • FIG. 4 shows functional blocks of the information processing apparatus 10.
  • the information processing apparatus 10 includes a sensor data receiving unit 20, a captured image receiving unit 22, an input data receiving unit 24, a display control unit 50, and a communication unit 46 as an input / output interface with the outside.
  • the information processing apparatus 10 further includes an HMD information acquisition unit 30, a gaze direction detection unit 32, a reception unit 34, a main image generation unit 42, an additional image generation unit 44, and an attention image detection unit 48.
  • each element described as a functional block that performs various processing can be configured in hardware as a circuit block, a memory, or another LSI, and software can be loaded in a memory. It is realized by a program etc. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any of them.
  • a computer program including a plurality of modules corresponding to a plurality of functional blocks in FIG. 4 may be installed in a storage (for example, the storage device 13) of the information processing apparatus 10.
  • the CPU or GPU of the information processing apparatus 10 may exert the functions of the respective functional blocks in FIG. 4 by reading out the computer program to the main memory and executing it.
  • the main image generation unit 42 determines an aspect of the game image for the left eye and the game image for the right eye as the image of the main content.
  • the display control unit 50 provides each game image to the HMD 100 and causes the display panel 130 to display the game image.
  • the sensor data reception unit 20 receives sensor data at a predetermined cycle from the posture sensor 124 of the HMD 100 worn by the user, and supplies the sensor data to the HMD information acquisition unit 30.
  • the photographed image reception unit 22 receives an image obtained by photographing the HMD 100 at a predetermined cycle from the imaging device 14, and supplies the image to the HMD information acquisition unit 30.
  • the imaging device 14 captures the forward space every (1/60) seconds, and the captured image receiving unit 22 receives a captured image every (1/60) seconds.
  • the HMD information acquisition unit 30 acquires sensor data and a photographed image as posture information indicating the posture of the HMD 100 mounted on the head of the user.
  • the HMD information acquisition unit 30 supplies the acquired orientation information to the gaze direction detection unit 32.
  • the gaze direction detection unit 32 detects the gaze direction of the user according to the posture information of the HMD 100, in other words, determines the gaze direction of the user.
  • the HMD information acquisition unit 30 detects a change in attitude of the HMD 100 from the sensor data of the attitude sensor 124.
  • the HMD information acquisition unit 30 may specify the posture change of the HMD 100 from sensor data of the three-axis gyro sensor.
  • the HMD information acquisition unit 30 further utilizes the imaging result of the tracking light emission marker 110 to enhance the detection accuracy of the posture change.
  • the gaze direction detection unit 32 provides the detected gaze direction to the main image generation unit 42, the attention image detection unit 48, and the display control unit 50.
  • the input data receiving unit 24 receives key data input by the user from the input device 16 and supplies the key data to the receiving unit 34.
  • the accepting unit 34 provides key data to the main image generating unit 42 as operation information for the game.
  • the main image generation unit 42 uses the gaze direction provided from the gaze direction detection unit 32 as operation information that determines the gaze of the user character. Further, the main image generation unit 42 uses the key data supplied from the reception unit 34 as operation information for determining the motion of the user character.
  • the process of moving the character in the virtual three-dimensional space by the operation of the input device 16 may use a known technique.
  • the main image generation unit 42 moves the user character in the virtual three-dimensional space, and performs rendering processing of the virtual three-dimensional space based on the position (virtual three-dimensional space coordinate value) of the user character and the gaze direction.
  • the main image generation unit 42 arranges a virtual camera whose optical axis is the gaze direction of the user character at the position of the user character, and acquires an image of the virtual camera, thereby displaying the virtual three-dimensional image displayed on the HMD 100 A game image of space may be generated.
  • the user moves the character by key input of the input device 16 and changes the direction of the HMD 100 to change the gaze direction of the character.
  • the user can move to 360 degrees and view 360 degrees, and in that sense, the HMD 100 is used as a display device capable of displaying all directions of the game virtual space.
  • the communication unit 46 communicates with the server 3 via the AP 17.
  • the communication unit 46 of the embodiment receives the data of the advertisement transmitted from the server 3.
  • the communication unit 46 may receive the image data of the advertisement itself from the server 3.
  • the notable image detection unit 48 is an advertisement having a high degree of matching of the display position in the HMD 100 with the user's gaze direction among the plurality of advertisement images displayed on the HMD 100 based on the gaze direction provided from the gaze direction detection unit 32.
  • An image is detected as an image of interest.
  • the attention image detection unit 48 may measure, for each of the plurality of advertisement images displayed on the HMD 100, a time (hereinafter also referred to as “attention time”) that matches the user's gaze direction.
  • the attention image detection unit 48 may detect, as an attention image, an advertisement image in which the attention time is equal to or more than a predetermined threshold.
  • An appropriate value may be determined as the threshold based on the knowledge and experience of the developer, an experiment using the information processing system 1, and the like. For example, the threshold may be 5 seconds.
  • the additional image generation unit 44 generates an advertisement image to be displayed on the HMD 100 as an image of the additional content, based on the data of the advertisement stored in advance in the information processing apparatus 10 or the data of the advertisement transmitted from the server 3. .
  • the additional image generation unit 44 outputs the generated data of the advertisement image to the display control unit 50.
  • the additional image generation unit 44 selects advertisement data to be displayed from among data of a plurality of types of advertisements, and based on the selected data, selects an advertisement image for promoting a specific product or service. It may be generated.
  • the additional image generation unit 44 may output the advertisement image provided from the server 3 to the display control unit 50 as it is.
  • the display control unit 50 transmits the game image generated by the main image generation unit 42 to the HMD 100, and causes the HMD 100 to display the game image. At the same time, the display control unit 50 transmits the advertisement image generated by the additional image generation unit 44 to the HMD 100, and causes the HMD 100 to display the advertisement image.
  • the display control unit 50 determines the display position of the advertisement image in the HMD 100, in other words, determines the display position (for example, coordinates etc.) of the advertisement image in the virtual three-dimensional space displayed on the HMD 100.
  • the display control unit 50 controls the HMD 100 to display an advertisement image at the display position in the HMD 100, in other words, the display position in the virtual three-dimensional space.
  • the display control unit 50 may display an advertisement image so as to be superimposed on a partial area of the game image.
  • the display control unit 50 controls the display mode of the advertisement image in the HMD 100 according to the gaze direction of the user detected by the gaze direction detection unit 32.
  • the display control unit 50 may change the product or service that the advertisement image is targeted for sales promotion or recognition promotion, in other words, the product or service indicated by the advertisement image.
  • the display control unit 50 may change the display presence / absence or the display position of the advertisement image in the HMD 100 according to the direction of the user's line of sight.
  • the display control unit 50 may execute a plurality of types of control processing on the display mode of the advertisement image in parallel.
  • the display control unit 50 may hide the advertisement image displayed until then, when the state in which the line of sight direction of the user matches the display position of the advertisement image in the HMD 100 continues for a predetermined time or more.
  • the display control unit 50 may transmit the game image from which the advertisement image is excluded to the HMD 100 for display.
  • An appropriate value may be determined based on the developer's knowledge and experience, an experiment using the information processing system 1, and the like as the predetermined time which is a threshold value for not displaying the advertisement image.
  • the predetermined time may be 5 seconds.
  • FIG. 5A and 5B schematically show images displayed on the HMD 100.
  • the display control unit 50 sets the upper center of the screen of the HMD 100 as the initial display position of the advertisement image 202.
  • the display control unit 50 may display the advertisement image 202 in the upper central region of the game image 200 when the advertisement image 202 is initially displayed.
  • the display control unit 50 may store in advance correspondence relationships between a plurality of types of gaze directions of the user and a plurality of areas of the screen of the HMD 100. When a specific gaze direction is detected, the display control unit 50 may specify an area of the screen corresponding to the gaze direction (that is, an area viewed by the user). The display control unit 50 is in a state in which the line of sight direction of the user matches the display position (in other words, the display area) of the advertisement image 202. In the example of FIG. 5A, the line of sight direction of the user is the upper center of the screen of the HMD 100 The time at which the state of pointing to has continued (hereinafter also referred to as “attention time”) may be measured.
  • the display control unit 50 may switch the advertisement image 202 to non-display when the attention time to the advertisement image 202 being displayed becomes equal to or more than the above threshold.
  • FIG.5 (b) has shown the state by which the advertisement image 202 of Fig.5 (a) was switched to non-display.
  • the advertisement when it is confirmed that the user has seen an advertisement in the virtual three-dimensional space for a certain amount of time, the advertisement is automatically deleted. This makes it easy to avoid making the user feel bothersome by the advertisement image while making the user recognize the advertisement target.
  • the additional image generation unit 44 may generate moving image data (in other words, a moving image advertisement) as an advertisement image, or may obtain moving image data as an advertisement image from the server 3.
  • the display control unit 50 may start reproduction of the moving image advertisement when the line-of-sight direction of the user matches the display position of the advertisement image on the HMD 100. For example, the display control unit 50 sequentially transmits a plurality of advertisement images corresponding to a plurality of elapsed times from the start of reproduction (in other words, a plurality of advertisement images of the moving image reproduction result) to the HMD 100, and sequentially transmits the plurality of advertisement images to the HMD 100 It may be displayed.
  • the display control unit 50 may suppress the reproduction of the moving image advertisement when the line-of-sight direction of the user does not match the display position of the advertisement image in the HMD 100.
  • the display control unit 50 causes the time (the above-mentioned attention time) during which the state in which the user's gaze direction matches the display position of the moving image advertisement in the HMD 100 continues becomes a predetermined threshold (for example, 2 seconds) or more If so, playback of the video advertisement may be started.
  • the display control unit 50 may suppress the reproduction of the moving image advertisement if the attention time is less than the predetermined threshold.
  • the display control unit 50 may control the HMD 100 to hide the moving image advertisement displayed until then, when the reproduction of the moving image advertisement ends.
  • FIG. 6A and 6B schematically show images displayed on the HMD 100.
  • the display control part 50 of the Example displays the moving image advertisement 204 in a still state at the time of the initial display of the moving image advertisement 204.
  • the display control unit 50 may display a still image indicating the first frame of the moving image advertisement 204 (in other words, an image displayed at the start of reproduction).
  • the display control unit 50 starts reproduction and display of the video advertisement 204 when the user's gaze direction matches the display position of the video advertisement 204, for example, when the user's gaze direction points to the upper center of the screen of the HMD 100.
  • FIG. 6B shows a state in which the moving image advertisement 204 being reproduced is displayed.
  • the video advertisement is constantly played back, there is a possibility that the visual of the game image which is the main content is hindered.
  • communication traffic will increase if the video advertisement is constantly reproduced.
  • the reproduction of the video advertisement is started when the user sees the advertisement in the virtual three-dimensional space, the video advertisement is effectively presented to the user while avoiding the above-mentioned adverse effect. Can.
  • the display control unit 50 may switch the content of the advertisement image to a content different from that before when the state in which the user's gaze direction matches the display position of the advertisement image in the HMD 100 continues for a predetermined time or more. In this switching process, the display control unit 50 ends the display of the second image advertising the first thing and starts the display of the second image advertising the second thing different from the first thing. May be An appropriate value may be determined based on the knowledge and experience of the developer, an experiment using the information processing system 1, and the like as the predetermined time which is a threshold for changing the content of the advertisement image. For example, the predetermined time may be 5 seconds.
  • FIGS. 7A and 7B schematically show images displayed on the HMD 100.
  • the display control unit 50 according to the embodiment displays the first advertisement image 206 at the upper center of the screen of the HMD 100.
  • the first advertisement image 206 includes, for example, content for advertising a first thing.
  • the display control unit 50 is in a state in which the gaze direction of the user matches the display position of the first advertisement image 206, that is, the time in which the gaze direction of the user continues pointing to the upper center of the screen of the HMD 100 (focused time) May be measured.
  • the display control unit 50 replaces the first advertisement image 206 when the attention time for the first advertisement image 206 is equal to or more than the above threshold, and a second advertisement image for advertising contents different from the first one. 208 may be displayed.
  • FIG. 7B shows a state in which the display target in the HMD 100 is changed from the first advertisement image 206 to the second advertisement image 208.
  • the effectiveness of the advertisement may be weak, for example, when the user has other interests.
  • the advertisement when a user views an advertisement for a certain period of time, the advertisement is replaced with another advertisement. This makes it possible to effectively present advertisements to many users having various interests and values.
  • the display control unit 50 may cause the HMD 100 to display a plurality of advertisement images indicating a plurality of types of content.
  • the type of content that can be set in the advertisement image can be said to be an advertisement target category, and may be classified into, for example, a game, a concert, a television program, an event, and the like.
  • the plurality of advertisement images include at least two advertisement images different in a thing or service targeted for advertisement.
  • the plurality of advertisement images may include, for example, a first advertisement image for advertising a game, a second advertisement image for advertising a concert, and a third advertisement image for advertising a television program.
  • the display control unit 50 sets a plurality of types of content indicated by the target image among a plurality of types of contents indicated by the plurality of advertisement images.
  • the priority may be set to the advertisement image of.
  • the display control unit 50 may set the content of the type indicated by the image of interest for all or a large number of advertisement images among the plurality of advertisement images displayed simultaneously on the HMD 100.
  • FIG. 8A and FIG. 8B schematically show images displayed on the HMD 100.
  • the display control unit 50 causes the HMD 100 to display an advertisement image 210a (for example, a game), an advertisement image 210b (for example, a concert), and an advertisement image 210c (for example, a television program) having different advertisement targets.
  • the focused image detection unit 48 detects the advertisement image 210b as a focused image.
  • the display control unit 50 sets an advertisement target category indicated by the advertisement image 210b of FIG. 8A in a plurality of advertisement images.
  • the display control unit 50 changes all of the three advertisement images into advertisement images 210b (for example, advertisement images indicating three concerts).
  • the display control unit 50 displays an advertisement image so as to display an advertisement image in an area in the display area of the HMD 100 that matches the user's gaze direction (also referred to as the gaze center) detected by the gaze direction detection unit 32. You may change the position. For example, the display control unit 50 may specify an area that matches the user's gaze direction among a plurality of display areas obtained by dividing the screen of the HMD 100. The display control unit 50 may change the display position of the advertisement image so as to display the advertisement image in the specified area when the area currently displaying the advertisement image is different from the specified area.
  • FIGS. 9A and 9B schematically show images displayed on the HMD 100.
  • the display control unit 50 sets the upper center of the screen of the HMD 100 as the initial display position of the advertisement image 202.
  • the display control unit 50 may display the advertisement image 202 in the upper central region of the game image 200 when the advertisement image 202 is initially displayed.
  • the display control unit 50 changes the display position of the advertisement image 202 from the upper center to the central right portion. For example, the display control unit 50 may start the display of the advertisement image 202 in the center right portion of the screen of the HMD 100 while ending the display of the advertisement image 202 in the upper center of the screen of the HMD 100.
  • the display control unit 50 changes the display position of the advertisement image so that the advertisement image is displayed in the specific area when the sight line direction of the user stays in the specific area of the screen of the HMD 100 for a predetermined time or more.
  • the display control unit 50 may determine the area as the display position of the advertisement image, when the area in the screen matching the user's gaze direction does not change for a predetermined time or more.
  • An appropriate value may be determined based on the knowledge and experience of the developer, an experiment using the information processing system 1, and the like as the predetermined time described above.
  • the predetermined time may be 3 seconds. According to this modification, it is possible to arrange the advertisement image at a position where the user surely sees, and to suppress the frequency of changing the display position of the advertisement image.
  • the display control unit 50 causes the advertisement image to be displayed in an area different from the area in the display area of the HMD 100 that matches the user's gaze direction (also referred to as the gaze center) detected by the gaze direction detection unit 32.
  • the display position of the advertisement image may be changed.
  • An area different from the area matching the user's line-of-sight direction can also be said to be an area corresponding to the outer periphery (also referred to as the outer edge) of the user's line-of-sight direction.
  • the area is the area avoiding the user's gaze direction. If the advertisement image is being displayed in a region different from the region that matches the user's gaze direction, the display control unit 50 may maintain the display position of the advertisement image as it is.
  • FIGS. 10A and 10B schematically show images displayed on the HMD 100.
  • the display control unit 50 sets the upper center of the screen of the HMD 100 as the initial display position of the advertisement image 202.
  • the gaze direction detected by the gaze direction detection unit 32 matches the center of the screen of the HMD 100.
  • the display control unit 50 changes the display position of the advertisement image 202 to a position avoiding the screen center. For example, as shown in FIG. 10 (b), it may be changed to the peripheral portion of the screen (in other words, positions near four corners).
  • the user may be disgusted.
  • the advertisement image is displayed and the image of the main content is difficult to see, the user may be disgusted.
  • the advertisement image is displayed and the image of the main content is difficult to see, the user may be disgusted.
  • the advertisement image by arranging the advertisement image avoiding the line-of-sight center, it is possible to gradually foster the user's interest in the target of the advertisement while avoiding giving the user a disgust.
  • the HMD 100 captures an image of the user's eye with a camera, and detects a user's gaze direction based on the captured image (referred to as a "gaze sensor” herein). It may be built in.
  • the HMD 100 may transmit, to the information processing apparatus 10, the line-of-sight sensor data indicating the line-of-sight direction of the user, which is output from the line-of-sight sensor.
  • the sensor data receiving unit 20 of the information processing device 10 may receive line-of-sight sensor data, and the HMD information acquisition unit 30 may output posture information including the line-of-sight sensor data to the line-of-sight direction detecting unit 32.
  • the gaze direction detection unit 32 may detect the gaze direction of the user based on gaze sensor data included in the posture information.
  • the information processing apparatus 10 controls the display mode of the additional content image (advertisement image) in the HMD 100 in accordance with the line-of-sight direction of the user wearing the HMD 100.
  • the information processing apparatus 10 includes a head direction detection unit that detects the direction of the head of the user based on the posture information acquired by the HMD information acquisition unit 30, instead of the gaze direction detection unit 32. It is also good.
  • the orientation of the user's head may be an angle from a predetermined reference orientation.
  • the main image generation unit 42, the attention image detection unit 48, and the display control unit 50 of the information processing apparatus 10 execute the processing described in the embodiment according to the direction of the head of the user instead of the gaze direction of the user. It is also good.
  • the information processing apparatus 10 determines the presence / absence, the type, and the position of the advertisement image on the screen of the HMD 100.
  • the HMD 100 may execute this determination process.
  • the server 3 may execute this determination process.
  • the HMD 100 or the server 3 may have the function of the information processing apparatus 10 described in the embodiment (for example, FIG. 4).
  • the HMD 100 or the server 3 may have the functions of the HMD information acquisition unit 30, the gaze direction detection unit 32, the additional image generation unit 44, the attention image detection unit 48, and the display control unit 50 described in the embodiment.
  • the information processing apparatus 10 selects, from among a plurality of areas in the game image 200 which is the main content, an area that matches the user's gaze direction or head direction You may further provide the attention area detection part detected as these.
  • the display control unit 50 may switch the content of the advertisement image 202 to the content corresponding to the attention area.
  • the data of the game image 200 displayed on the HMD 100 includes, for at least one of a plurality of areas into which the game image 200 is divided, identification data indicating an object (person, thing, character, etc.) drawn in the area. It corresponds beforehand. Identification data of an object present in each area may be associated with each frame of the game image 200 or each group obtained by collecting a predetermined number of frames.
  • the above-mentioned attention area detection unit detects, as an attention area, an area that matches the line-of-sight direction of the user detected by the line-of-sight direction detection unit 32 among a plurality of areas obtained by dividing the game image 200.
  • the attention area detection unit may store correspondences between a plurality of types of gaze directions and a plurality of screen areas.
  • the attention area detection unit may specify a screen area that matches the viewing direction, and detect an area of the game image 200 displayed in the specified screen area as an attention area.
  • the information processing apparatus 10 further includes a preference detection unit that detects the preference of the user according to the attention area detected by the attention area detection unit.
  • the preference detection unit detects an object associated with the attention area detected by the attention area detection unit as one that indicates the user's preference, in other words, one that matches the user's preference.
  • Identification information of an object that can be displayed on the game image 200 is associated with each of the plurality of advertisement images 202 in advance.
  • the advertisement image 202 associated with an object may display advertisement content including the object.
  • the display control unit 50 is the advertisement image 202 that matches the user's preference detected by the preference detection unit, and, for example, selects the advertisement image 202 corresponding to the object in the attention area and causes the HMD 100 to display it.
  • the HMD 100 it is possible to cause the HMD 100 to display an advertisement that further matches the user's preference as additional content.
  • the main content is a video of a concert and a video in which a plurality of performers are simultaneously displayed, it is possible to detect which performer the user pays attention to and change the advertisement according to the performer paying attention .
  • the effect of the advertisement can be further enhanced by displaying an advertisement for promoting goods or services, which is an advertisement associated with a performer the user is focusing on.
  • 10 information processing apparatus 32 gaze direction detection unit, 42 main image generation unit, 44 additional image generation unit, 48 target image detection unit, 50 display control unit, 100 HMD.
  • the present invention can be applied to an apparatus or system that controls the display of a head mounted display.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Marketing (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Software Systems (AREA)
  • Optics & Photonics (AREA)
  • Computer Graphics (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biophysics (AREA)
  • Cardiology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Controls And Circuits For Display Device (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2018/030421 2017-08-23 2018-08-16 情報処理装置および画像表示方法 Ceased WO2019039378A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/631,568 US11231587B2 (en) 2017-08-23 2018-08-16 Information processing apparatus and image display method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2017160554A JP6845111B2 (ja) 2017-08-23 2017-08-23 情報処理装置および画像表示方法
JP2017-160554 2017-08-23

Publications (1)

Publication Number Publication Date
WO2019039378A1 true WO2019039378A1 (ja) 2019-02-28

Family

ID=65438633

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/030421 Ceased WO2019039378A1 (ja) 2017-08-23 2018-08-16 情報処理装置および画像表示方法

Country Status (3)

Country Link
US (1) US11231587B2 (https=)
JP (2) JP6845111B2 (https=)
WO (1) WO2019039378A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114041101A (zh) * 2019-07-11 2022-02-11 惠普发展公司，有限责任合伙企业 用于显示器的眼睛跟踪

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2020195521A1 (ja) 2019-03-27 2020-10-01 パナソニックＩｐマネジメント株式会社 ヘッドマウントディスプレイ
JP2020166555A (ja) * 2019-03-29 2020-10-08 株式会社三井住友銀行 Ａｒプラットフォームシステム、方法およびプログラム
US11106914B2 (en) 2019-12-02 2021-08-31 At&T Intellectual Property I, L.P. Method and apparatus for delivering content to augmented reality devices
JP6905615B1 (ja) * 2020-03-31 2021-07-21 エイベックス・テクノロジーズ株式会社 拡張現実システム
JP2021163496A (ja) * 2020-03-31 2021-10-11 エイベックス・テクノロジーズ株式会社 拡張現実システム
JP7455651B2 (ja) * 2020-04-27 2024-03-26 キヤノン株式会社 電子機器及びその制御方法
JP6932224B1 (ja) 2020-06-01 2021-09-08 株式会社電通 広告表示システム
US11756081B2 (en) * 2020-06-12 2023-09-12 International Business Machines Corporation Rendering privacy aware advertisements in mixed reality space
CN116615704A (zh) * 2020-09-24 2023-08-18 苹果公司 用于姿势检测的头戴式设备
JP7578904B2 (ja) * 2020-11-30 2024-11-07 マツダ株式会社 広告提供システム及び広告提供方法
JP7589036B2 (ja) * 2020-12-08 2024-11-25 キヤノン株式会社 情報処理装置、情報処理システム、情報処理方法及びプログラム
KR102665453B1 (ko) 2022-01-17 2024-05-10 엔에이치엔 주식회사 시선인식 기반 맞춤형 콘텐츠를 제공하는 장치 및 방법
JP2023105524A (ja) * 2022-01-19 2023-07-31 株式会社ソニー・インタラクティブエンタテインメント 表示制御装置、ヘッドマウントディスプレイ、および表示制御方法
WO2023250096A1 (en) * 2022-06-23 2023-12-28 Universal City Studios Llc Interactive imagery systems and methods
US12254717B2 (en) * 2022-06-23 2025-03-18 Universal City Studios Llc Interactive imagery systems and methods
JP7257080B1 (ja) 2022-07-06 2023-04-13 株式会社PocketRD 映像分析装置、映像分析方法及び映像分析プログラム
WO2024090844A1 (ko) * 2022-10-28 2024-05-02 삼성전자주식회사 화면의 상태를 변경하기 위한 웨어러블 장치 및 그 방법
WO2026013804A1 (ja) * 2024-07-10 2026-01-15 マクセル株式会社 情報表示装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008033891A (ja) * 2006-06-27 2008-02-14 Matsushita Electric Ind Co Ltd 表示装置及びその制御方法
US20150130705A1 (en) * 2013-11-12 2015-05-14 Samsung Electronics Co., Ltd. Method for determining location of content and an electronic device
JP2016126773A (ja) * 2014-12-29 2016-07-11 イマージョン コーポレーションＩｍｍｅｒｓｉｏｎ Ｃｏｒｐｏｒａｔｉｏｎ 視線追跡に基づく触覚効果を生成するシステム及び方法
JP2017040970A (ja) * 2015-08-17 2017-02-23 株式会社コロプラ ヘッドマウントディスプレイシステムを制御する方法、および、プログラム
JP6126271B1 (ja) * 2016-05-17 2017-05-10 株式会社コロプラ 仮想空間を提供する方法、プログラム及び記録媒体

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7928926B2 (en) * 2006-06-27 2011-04-19 Panasonic Corporation Display apparatus and method for hands free operation that selects a function when window is within field of view
US8879155B1 (en) * 2011-11-09 2014-11-04 Google Inc. Measurement method and system
KR101479471B1 (ko) * 2012-09-24 2015-01-13 네이버 주식회사 사용자의 시선에 기초하여 광고를 제공하는 방법 및 시스템
JP5951465B2 (ja) * 2012-12-19 2016-07-13 Kddi株式会社 道路情報に基づいて見通しエリアを決定するプログラム、情報端末、サーバ及び方法
US9335547B2 (en) * 2013-03-25 2016-05-10 Seiko Epson Corporation Head-mounted display device and method of controlling head-mounted display device
JP6492419B2 (ja) * 2014-05-23 2019-04-03 セイコーエプソン株式会社 頭部装着型表示装置、頭部装着型表示装置を制御する方法、コンピュータープログラム、画像表示システム、および、情報処理装置
JP6705124B2 (ja) * 2015-04-23 2020-06-03 セイコーエプソン株式会社 頭部装着型表示装置、情報システム、頭部装着型表示装置の制御方法、および、コンピュータープログラム
US10539797B2 (en) 2016-05-06 2020-01-21 Colopl, Inc. Method of providing virtual space, program therefor, and recording medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008033891A (ja) * 2006-06-27 2008-02-14 Matsushita Electric Ind Co Ltd 表示装置及びその制御方法
US20150130705A1 (en) * 2013-11-12 2015-05-14 Samsung Electronics Co., Ltd. Method for determining location of content and an electronic device
JP2016126773A (ja) * 2014-12-29 2016-07-11 イマージョン コーポレーションＩｍｍｅｒｓｉｏｎ Ｃｏｒｐｏｒａｔｉｏｎ 視線追跡に基づく触覚効果を生成するシステム及び方法
JP2017040970A (ja) * 2015-08-17 2017-02-23 株式会社コロプラ ヘッドマウントディスプレイシステムを制御する方法、および、プログラム
JP6126271B1 (ja) * 2016-05-17 2017-05-10 株式会社コロプラ 仮想空間を提供する方法、プログラム及び記録媒体

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114041101A (zh) * 2019-07-11 2022-02-11 惠普发展公司，有限责任合伙企业 用于显示器的眼睛跟踪
EP3973372A4 (en) * 2019-07-11 2023-01-11 Hewlett-Packard Development Company, L.P. EYE TRACKING FOR SCREENS

Also Published As

Publication number Publication date
JP2019039988A (ja) 2019-03-14
JP2021103303A (ja) 2021-07-15
US20200201048A1 (en) 2020-06-25
US11231587B2 (en) 2022-01-25
JP6845111B2 (ja) 2021-03-17

Similar Documents

Publication Publication Date Title
JP6845111B2 (ja) 情報処理装置および画像表示方法
JP6511386B2 (ja) 情報処理装置および画像生成方法
JP6615732B2 (ja) 情報処理装置および画像生成方法
JP6058184B1 (ja) ヘッドマウントディスプレイシステムを制御するための方法、および、プログラム
US11373379B2 (en) Image generation apparatus and image generation method for generating augmented reality images based on user interaction
JP6807455B2 (ja) 情報処理装置および画像生成方法
US12278936B2 (en) Information processing system, information processing method, and computer program
US11647354B2 (en) Method and apparatus for providing audio content in immersive reality
JP2019087226A (ja) 情報処理装置、情報処理システムおよび表情画像出力方法
JP6126271B1 (ja) 仮想空間を提供する方法、プログラム及び記録媒体
JP2019046291A (ja) 情報処理装置および画像表示方法
US20220148253A1 (en) Image rendering system and method
JP6518645B2 (ja) 情報処理装置および画像生成方法
JP6921204B2 (ja) 情報処理装置および画像出力方法
JP2017208808A (ja) 仮想空間を提供する方法、プログラム及び記録媒体
JP2020174329A (ja) 情報処理装置、表示方法およびコンピュータプログラム
JP2017162443A (ja) ヘッドマウントディスプレイシステムを制御するための方法、および、プログラム
JP7733481B2 (ja) 情報処理装置および画像生成方法
JP2024031113A (ja) 情報処理装置および画像生成方法
JP2022015647A (ja) 情報処理装置および画像表示方法
US20250303290A1 (en) Information processing apparatus and image generation method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18847300

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18847300

Country of ref document: EP

Kind code of ref document: A1