WO2019186622A1 - Display device, display method, program, and non-transitory computer-readable information recording medium - Google Patents

Display device, display method, program, and non-transitory computer-readable information recording medium Download PDF

Info

Publication number
WO2019186622A1
WO2019186622A1 PCT/JP2018/012034 JP2018012034W WO2019186622A1 WO 2019186622 A1 WO2019186622 A1 WO 2019186622A1 JP 2018012034 W JP2018012034 W JP 2018012034W WO 2019186622 A1 WO2019186622 A1 WO 2019186622A1
Authority
WO
WIPO (PCT)
Prior art keywords
display device
line
image
virtual space
change
Prior art date
Application number
PCT/JP2018/012034
Other languages
French (fr)
Japanese (ja)
Inventor
裕章 岩瀬
Original Assignee
楽天株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 楽天株式会社 filed Critical 楽天株式会社
Priority to PCT/JP2018/012034 priority Critical patent/WO2019186622A1/en
Publication of WO2019186622A1 publication Critical patent/WO2019186622A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/66—Transforming electric information into light information

Definitions

  • the present invention relates to a display device, a display method, a program, and a non-transitory computer-readable information recording medium.
  • Patent Document 1 captures a driver's blind spot area with a camera and captures the captured image as a driver's viewpoint.
  • a method is disclosed in which a translucent image of a vehicle viewed from the driver's viewpoint is superimposed and displayed.
  • the direction of the user wearing the head mounted display is detected, and the line of sight to the three-dimensional space is detected according to the detected orientation.
  • the blind spot area of the driver is assumed to be an area that the driver pays attention in advance, and the image of the blind spot area is displayed superimposed on the translucent image of the vehicle.
  • an object to which a user pays attention is not predetermined. Even in such a case, there is a need for a technique for identifying an object that is noticed by the user.
  • the present invention is to solve the above-described problems, and provides a display device, a display method, a program, and a non-transitory computer-readable information recording medium that can identify an object that a user pays attention to. With the goal.
  • a display device provides: A display device, A detection unit for detecting a change in orientation of the display device in real space; A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction; A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction; A display unit for displaying the generated image on a screen of the display device; Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest A specific part to be It is characterized by providing.
  • the immediately preceding position in the image generated immediately before the change of the line-of-sight direction of the position where each object exists in the virtual space immediately before the line-of-sight direction is changed; A position immediately after in the image to be generated immediately after the line-of-sight direction is changed; Seeking A difference between the immediately following position and the immediately preceding position may be used as the velocity component of each object.
  • the object having the minimum velocity component may be small on the predetermined criterion.
  • An object whose speed component is equal to or less than a predetermined threshold may be small according to the predetermined reference.
  • An object whose velocity component is less than a predetermined threshold may be small according to the predetermined reference.
  • An object whose ratio of the velocity component to the detected change in orientation is less than a predetermined threshold may be small on the predetermined basis.
  • the changing unit changes an observation position in a direction opposite to a direction in which the line-of-sight direction has changed, centering on an object having the smallest velocity component among the objects of interest in accordance with a change in the line-of-sight direction
  • the generation unit may generate an image representing a state in which the virtual space is observed in the viewing direction from the observation position.
  • the change unit may change the observation position in proportion to a change amount in the line-of-sight direction.
  • the detection unit further detects the position of the display device
  • the change unit further changes the observation position in the virtual space according to the detected position
  • the generation unit may generate an image representing a state in which the virtual space is observed in the viewing direction from the observation position.
  • the image may be generated after increasing the transparency of an object other than the object of interest among the plurality of objects and hiding the object of interest when the virtual space is observed in the line-of-sight direction.
  • the display method is: A display method executed by a display device, A detection step of detecting a change in orientation of the display device in real space; A change step of changing a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction; A generation step of generating an image representing a state in which the virtual space is observed in the line-of-sight direction; A display step of displaying the generated image on a screen of the display device; Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest Specific steps to It is characterized by providing.
  • the program according to the third aspect of the present invention is: Computer A detection unit that detects a change in orientation of the display device in real space; A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction; A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction; A display unit for displaying the generated image on a screen of the display device; Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest A specific part to be It is made to function as.
  • a non-transitory computer-readable information recording medium is provided.
  • Computer A detection unit that detects a change in orientation of the display device in real space;
  • a changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
  • a generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction;
  • the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest Specific part to do, It is made to function as.
  • the above information recording medium can be distributed and sold independently of the computer.
  • the non-transitory information recording medium refers to a tangible information recording medium.
  • Non-temporary information recording media are, for example, compact disks, flexible disks, hard disks, magnetic disks, digital video disks, magnetic tapes, and semiconductor memories.
  • the transitory information recording medium refers to the transmission medium (propagation signal) itself.
  • the temporary recording medium is, for example, an electric signal, an optical signal, or an electromagnetic wave.
  • the temporary storage area is an area for temporarily storing data and programs, and is a volatile memory such as RAM (Random Access Memory), for example.
  • FIG. 1 is a schematic block diagram showing a hardware configuration of the display device 100 according to the present embodiment.
  • the display device 100 is configured by, for example, a head mounted display including various sensors and a control device.
  • a head mounted display may be constructed by attaching a smartphone, tablet computer, fablet, or the like to the attachment.
  • the display device 100 is realized by causing the smartphone or the like to execute a program for causing a computer such as a smartphone to function as each unit.
  • the display device 100 includes a control unit 101, a ROM (Read Only Memory) 102, a RAM 103, a display 104, a sensor unit 105, and an operation unit 106. 107 is connected.
  • ROM Read Only Memory
  • the control unit 101 includes, for example, a CPU (Central Processing Unit) and controls the entire display device 100.
  • a CPU Central Processing Unit
  • the ROM 102 is a non-volatile memory that stores programs and various data for the control unit 101 to control the entire display device 100.
  • the RAM 103 is a volatile memory for temporarily storing information generated by the control unit 101 and data necessary for generating the information.
  • the display 104 includes an LCD (Liquid Crystal Display), a backlight, and the like, and displays an image output from the control unit 101 under the control of the control unit 101, for example.
  • LCD Liquid Crystal Display
  • the sensor unit 105 includes an attitude sensor and an acceleration sensor, and detects the orientation of the display device 100.
  • the operation unit 106 includes input devices such as buttons, a keyboard, and a touch panel.
  • the operation unit 106 receives an operation input from the user of the display device 100 and outputs a signal corresponding to the received operation input to the control unit 101.
  • FIG. 2 is a top view illustrating a state before the direction of the display device 100 and the user 202 changes in the real space 201 (time t a ).
  • FIG. 4 is a top view showing a state of the virtual space 301 before the orientations of the display device 100 and the user 202 change in the real space 201 (time t a ).
  • FIG. 3 is a top view showing a state after the orientations of the display device 100 and the user 202 are changed in the real space 201 (time t b ).
  • FIG. 2 is a top view illustrating a state before the direction of the display device 100 and the user 202 changes in the real space 201 (time t a ).
  • FIG. 4 is a top view showing a state of the virtual space 301 before the orientations of the display device 100 and the user 202 change in the real space 201 (time t a ).
  • FIG. 3 is a top view showing a state after the orientations of the display device 100 and the user 202
  • FIG. 5 is a top view showing a state of the virtual space 301 after the orientations of the display device 100 and the user 202 are changed in the real space 201 (time t b ).
  • time t b time t b
  • the user 202 wears the display device 100 in front of the eyes.
  • the display direction 203 of the display device 100 is opposite to the line-of-sight direction in which the user 202 looks at the display device 100.
  • a plurality of objects 401 to 404 that are stationary or moving are arranged in the virtual space 301.
  • the objects 402 and 403 are stationary, and the object 401 moves in the virtual space 301 in the direction of arrow A and the object 404 moves in the direction of arrow B, respectively.
  • the objects 401 to 404 are created by computer graphics, and the position, size, shape, and movement speed of the objects 401 and 404 in the virtual space 301 are set in advance.
  • the display device 100 displays a station platform reproduced by computer graphics
  • the objects 401 and 404 are objects that move in the station platform, such as trains and people, and the objects 402 and 403 are columns.
  • An object that is stationary at the station platform such as a signboard or a signboard.
  • orientation 211 in the real space 201 and the orientation 311 in the virtual space 301 may be shifted or matched. In this embodiment, both are shifted.
  • FIG. 6 is a schematic block diagram showing a functional configuration of the display device 100 according to the embodiment of the present invention.
  • the display device 100 includes a detection unit 111, a change unit 112, a generation unit 113, a specification unit 114, a display unit 115, and a display 104.
  • the detection unit 111 detects a change in orientation of the display device 100 in the real space 201.
  • the detection unit 111 detects a change in the display direction 203 of the display device 100 with respect to the reference axis in the real space 201 from the measurement result by the sensor unit 105.
  • As a reference axis in the real space 201 it is typical to adopt the direction of gravity in the real space 201 or the vertical direction of the display 104 of the display device 100.
  • control unit 101 and the sensor unit 105 cooperate to function as the detection unit 111.
  • the changing unit 112 changes the line-of-sight direction 303 with respect to the virtual space 301 according to the orientation of the display device 100 detected by the detecting unit 111.
  • the changing unit 112 the change in the display direction 203 of the display device 100 which is detected by the detection unit 111 between the time t a to time t b, the rotation amount ⁇ around the reference axis in the real space 201 get.
  • the display direction 203a is from the display device 100a to the user 202a.
  • the display direction 203b is directed from the display device 100b to the user 202b.
  • the angle formed by the display direction 203a and the display direction 203b is the rotation amount ⁇ around the reference axis.
  • the rotation amount ⁇ corresponds to a so-called yaw angle.
  • the user 202 has rotated his head to the left.
  • Time interval from time t a to time t b may be determined depending on the application.
  • this time interval can be a refresh interval (for example, a vertical synchronization period) of the display 104 of the display device 100.
  • the process is repeated, the time t b at a repeating unit is interpreted as the time t a in the next repeating unit.
  • the changing unit 112 changes the line-of-sight direction 303a to the line-of-sight direction 303b by rotating the line-of-sight direction 303a around the reference axis of the observation position 302a in the virtual space 301 by the rotation amount ⁇ .
  • control unit 101 functions as the changing unit 112.
  • the generation unit 113 generates an image representing a state in which the virtual space 301 is observed from the observation position 302 in the line-of-sight direction 303. For example, the generation unit 113 generates an image when viewed from the observation position 302 in the line-of-sight direction 303 using a technique such as perspective projection based on a predetermined position and shape of the objects 401 to 404 in the virtual space 301. To do.
  • the specifying unit 114 is a speed component that is not caused by the movement of the object in the virtual space 301 among the in-image speeds at which the positions where the plurality of objects 401 to 404 are drawn in the image generated by the generation unit 113 move. However, an object that is small on a predetermined basis is identified as the object of interest.
  • Figure 7A is an image 501a representing a state of observation of the virtual space 301 from the viewing position 302 in the viewing direction 303 at time t a
  • FIG. 7B a virtual space 301 from the viewing position 302 at time t b in the viewing direction 303
  • An image 501b representing the observed state is shown.
  • the velocity component resulting from the change in the line-of-sight direction 303 is generated as the in-image speed.
  • the positions drawn in the images 501a and 501b move.
  • the position drawn in the generated images 501a and 501b moves with the speed component S2x resulting from the change in the line-of-sight direction 303 as the in-image speed V2.
  • the speed component S2x resulting from the change in the line-of-sight direction 303 as the in-image speed V2.
  • the line-of-sight direction 303 changes, the combined speed of the speed component due to the change in the line-of-sight direction 303 and the speed component due to its own movement is obtained.
  • the in-image speed the position to be drawn in the generated image moves.
  • the object 401 is drawn in the generated images 501a and 501b with the combined speed of the speed component S1x resulting from the change in the line-of-sight direction 303 and the speed component Ax resulting from the movement of the object 401 as the in-image speed V1.
  • Move the position In general, when a user is paying attention to a moving object, the user views the object in the field of view, that is, the center in the generated image, and changes the line-of-sight direction 303 as the object moves. It is done.
  • the speed component resulting from the change in the line-of-sight direction 303 is caused by the movement of the object of interest.
  • the velocity component that is not considered is considered to be smaller than other objects. Accordingly, in the present embodiment, an object having a small velocity component that does not result from the movement of the object on the basis of a predetermined criterion is identified as the object of interest.
  • the identifying unit 114 first sets the line-of-sight direction 303 at the position where each object 401 to 404 existed in the virtual space 301 immediately before the line-of-sight direction 303 is changed.
  • the immediately preceding position in the image generated immediately before the change and the immediately following position in the image to be generated immediately after the line-of-sight direction 303 is changed are obtained.
  • the specifying unit 114 calculates the immediately preceding position from the known positions and moving speeds of the objects 401 to 404 in the virtual space 301, the observation position 302, and the line-of-sight direction 303.
  • the specifying unit 114 calculates the immediately following position from the immediately preceding position, the observation position 302 after the change, and the line-of-sight direction 303 after the change. Then, the specifying unit 114 calculates the difference between the immediately following position and the immediately preceding position as a velocity component caused by a change in the line-of-sight direction 303, that is, a velocity component not caused by the movement of the object.
  • the specifying unit 114 sets the positions of the representative points (for example, the center point and the center of gravity) of the objects 401 to 404 as the positions of the objects 401 to 404.
  • the object 401 is assumed to be positioned in Pa.
  • a specific unit 114, Pa based on the observation position 302 and the line-of-sight direction 303, calculates the position Pia in Pa in the image 501a generated at time t a.
  • the specification unit 114 Pa, observation position at time t b 302, and based on the line-of-sight direction 303 at time t b, calculates the position Pib in Pa in the image 501b to be generated at time t b.
  • specification part 114 calculates the difference of Pib and Pia as the speed component S1x resulting from the change of the gaze direction 303.
  • FIG. That is, the specifying unit 114 calculates the difference between Pib and Pia when it is assumed that the object 401 is stationary, that is, the in-image speed as the speed component S1x caused by the change in the line-of-sight direction 303. In this way, the specifying unit 114 calculates the velocity component due to the change in the line-of-sight direction 303 for each of the objects 401 to 404 in the virtual space 301.
  • the specifying unit 114 determines whether or not the speed component that is not caused by the movement of the object is small on a predetermined basis.
  • the specifying unit 114 uses the velocity caused by the change in the line-of-sight direction 303 obtained for each of the objects 401 to 404. Among the components, it is determined that the minimum velocity component is small on a predetermined basis. Then, the specifying unit 114 specifies the objects 401 to 404 having the smallest velocity component due to the change in the line-of-sight direction 303 as the target object.
  • the specifying unit 114 specifies the object 401 as the target object when the speed component S1x related to the object 401 among the speed components resulting from the change in the line-of-sight direction 303 of the objects 401 to 404 is minimum.
  • the specifying unit 114 may determine that the velocity component is small on a predetermined criterion.
  • the specifying unit 114 may determine that the speed component is small on a predetermined basis.
  • the specifying unit 114 determines that the speed component is a predetermined reference May be determined to be small.
  • control unit 101 functions as the specifying unit 114.
  • the generating unit 113 When generating the image, the generating unit 113 hides the target object when the virtual space 301 is observed in the line-of-sight direction 303 among the objects 401 to 404 other than the target object specified by the specifying unit 114. Generate the image after increasing the transparency of the object. For example, the generation unit 113 draws the background with a predetermined background color (for example, black), and draws the objects in order from the object far from the observation position 302. When a pixel for drawing an object other than the target object is drawn in a color other than the background color, the generation unit 113 has already drawn the pixel in the color of the target object. It is determined that the pixel of the object of interest is hidden by the object.
  • a predetermined background color for example, black
  • the generation unit 113 determines that the other object is an object that hides the object of interest, increases the transparency of the other object, and then generates an image. For example, when the identifying unit 114 identifies the object 401 as the object of interest, when the generating unit 113 determines that the object 402 is an object that hides the object 401, the transparency of the object 402 is increased as illustrated in FIG. From this, an image 501b is generated.
  • the generation unit 113 can generate an image that allows the user to pay attention to the part of the object 401 hidden by the object 402 by increasing the transparency of the object 402. Note that when the target object is hidden by another target object, the generation unit 113 generates an image without increasing the transparency of the other target object that hides the target object.
  • control unit 101 functions as the generation unit 113.
  • the display unit 115 displays the image generated by the generation unit 113 on the display 104.
  • the display unit 115 the time t a image 501a as shown in FIG. 7A at the time t b image as shown in FIG. 9 at 501b, is displayed on the respective display 104.
  • control unit 101 functions as the display unit 115.
  • FIG. 10 is a flowchart showing the flow of display processing executed by the control unit 101 of the display device 100. This process is started, for example, when an instruction to start this process is received via the operation unit 106.
  • the display device 100 sets the observation position 302 and the line-of-sight direction 303 in a predetermined initial state in the virtual space 301 (step S101).
  • the display device 100 initializes the virtual space 301 (step S102). At the time of this initialization, for example, the display device 100 acquires and sets the position, shape, orientation, and appearance of an object arranged in the virtual space 301, or is arranged at infinity in the virtual space 301. Get and set the assumed background image.
  • the display device 100 detects the orientation of the display device 100, and acquires the rotation amount ⁇ that the display direction of the display 104 has rotated about the reference axis in the real space 201 (step S103). Since step S103 is repeatedly executed, the rotation amount ⁇ acquired here corresponds to the difference between the display direction 203 measured last time by a sensor or the like and the display direction 203 measured this time.
  • the display device 100 updates the line-of-sight direction 303 according to the acquired rotation amount ⁇ (step S104).
  • the display device 100 selects an object for which the velocity component resulting from the change in the line-of-sight direction 303 is not calculated for the objects 401 to 404 in the virtual space 301 (step S105).
  • the display device 100 calculates the immediately preceding position and the immediately following position of the object selected in Step S105 (Step S106).
  • the display device 100 calculates the difference between the immediately after position calculated in step S106 and the immediately preceding position as a velocity component caused by a change in the line-of-sight direction 303 of the selected object (step S107). Then, the display device 100 determines whether or not the velocity component due to the change in the line-of-sight direction 303 has been calculated for all the objects 401 to 404 (step S108). When the display device 100 has not calculated the speed component due to the change in the line-of-sight direction 303 for all the objects 401 to 404 (step S108; No), the display device 100 returns to the process of step S105.
  • step S108 When the display device 100 calculates the velocity component due to the change in the line-of-sight direction 303 for all the objects 401 to 404 (step S108; Yes), the object related to the minimum velocity component among the calculated velocity components is selected as the target object. (Step S109).
  • the display device 100 determines whether there is an object that hides the target object identified in step S109 (step S110). When there is no object that hides the object of interest (step S110; No), the display device 100 proceeds to the process of step S112.
  • step S110 When there is an object that hides the object of interest (step S110; Yes), the display device 100 increases the transparency of the object that hides the object of interest by a predetermined degree (step S111).
  • the display apparatus 100 produces
  • the display device 100 waits until a vertical synchronization interrupt occurs on the display 104 (step S113), and then transfers the generated image to the display 104 and presents it to the user (step S114).
  • the display apparatus 100 updates the state of the virtual space 301 (step S115). For example, when the virtual space 301 is configured by computer graphics that changes over time, the position and orientation of the object are updated with the speed, acceleration, angular velocity, angular acceleration, and the like set for the object. A physical simulation is performed or the object is deformed based on a predetermined condition.
  • step S103 The display apparatus 100 repeatedly executes the above-described process until an end instruction for the process is received via the operation unit 106, for example. In addition, since it waits in step S113, the repetition period of this process becomes a vertical synchronizing period.
  • the display device 100 changes the line-of-sight direction 303 in the virtual space 301 when the orientation of the display device 100 in the real space 201 changes, and the objects 401 to 300 in the virtual space 301 change.
  • the in-image speeds 404 an object having a small speed component that is not caused by the movement of the objects 401 to 404 is specified as a target object. Therefore, when the user is following the moving object in the virtual space 301, even if the display device 100 cannot detect its own position, the display device 100 can detect the change of its own direction, thereby detecting the object that the user is interested in. Can be identified.
  • the display device 100 when the identified object of interest is hidden by another object, the display device 100 generates an image after increasing the transparency of the object that hides the object of interest. Therefore, the display device 100 can make the visible object portion hidden by other objects visible.
  • the display device 100 may realize the peeping of the object of interest by changing the observation position 302 according to the change in the line-of-sight direction 303.
  • the attention object 401a when the attention object 401a is viewed from the observation position 302a in the line-of-sight direction 303a, a part of the attention object 401a is hidden by the object 404a.
  • the user rotates the head and moves the position of the head in the real space 201 in order to look into the portion of the attention object 401 a hidden by the object 404 a.
  • the display device 100 cannot detect its own position, the movement cannot be detected even if the user moves his / her head.
  • the display device 100 detects the rotation of the head, that is, the change in the direction of the display device 100, changes the line-of-sight direction 303a in the virtual space 301 according to the change in the direction, and changes the direction of the line-of-sight direction 303a.
  • the observation position 302a may be changed according to the change.
  • the changing unit 112 may change the observation position 302 in a direction opposite to the direction in which the line-of-sight direction 303 is changed in accordance with the change in the changed line-of-sight direction 303. For example detection, during the period from the time t a to time t b, when a user, as shown in FIG. 3 is rotated ⁇ counterclockwise head from the state shown in FIG. 2, the changing unit 112, the detection unit 111 For the change in the display direction 203 of the display device 100, the rotation amount ⁇ around the reference axis in the real space 201 is acquired. Then, as illustrated in FIG.
  • the changing unit 112 changes the line-of-sight direction 303a to the line-of-sight direction 303b by rotating the line-of-sight direction 303a by the rotation amount ⁇ around the reference axis of the observation position 302a in the virtual space 301. Further, the changing unit 112 rotates the observation position 302a about the position Pa of the target object 401a specified by the specifying unit 114 in the direction opposite to the direction in which the line-of-sight direction 303 has changed by the rotation amount ⁇ , thereby observing. The position 302a is changed to the observation position 302b. In the example shown in FIG.
  • the line-of-sight direction 303a changes to the line-of-sight direction 303b by rotating counterclockwise, and therefore the changing unit 112 rotates the observation position 302a clockwise by the rotation amount ⁇ .
  • the observation position 302b is changed.
  • the observation position 302 is set in the direction opposite to the direction in which the line-of-sight direction 303 is changed according to the changed line-of-sight direction 303.
  • the changing unit 112 selects an object having a minimum velocity component due to a change in the line-of-sight direction 303 among the plurality of attention objects, ie, a velocity component that does not result from its own movement.
  • the observation position 302 may be changed in the direction opposite to the direction in which the line-of-sight direction 303 is changed.
  • the amount by which the changing unit 112 rotates the observation position 302 in the direction opposite to the direction in which the line-of-sight direction 303 changes is not limited to the same amount as the rotation amount ⁇ in the line-of-sight direction 303.
  • the changing unit 112 may change the observation position 302 in a direction opposite to the direction in which the line-of-sight direction 303 is changed with a rotation amount proportional to the rotation amount ⁇ in the line-of-sight direction 303.
  • the display device 100 detects a change in the orientation of the display device 100 and identifies an object of interest based on the change.
  • the display device 100 may further have a function of detecting the position of the display device 100.
  • the changing unit 112 may change the observation position 302 in the virtual space 301 according to the detected position of the display device 100.
  • the changing unit 112 the change in the position 204 of the display device 100 which is detected by the detection unit 111 between the time t a to time t b, obtains the movement amount d.
  • the representative point for example, the center point or the center of gravity point
  • the representative point of the display device 100 is located at the position 204a.
  • the representative point of the display device 100 is located at the position 204b. Therefore, the movement amount d of the display device 100 is a difference vector between the position 204a and the position 204b. Then, as illustrated in FIG.
  • the changing unit 112 changes the observation position 302 a to the observation position 302 b by moving the observation position 302 a by the movement amount d in the virtual space 301.
  • the generation unit 113 generates an image representing a state in which the virtual space 301 is observed from the observation position 302b in the visual line direction 303b.
  • the specifying unit 114 has a predetermined speed component that is not caused by the movement of the object among the in-image speeds at which the position where the object is drawn in the image generated by the generating unit 113 moves.
  • a small object is identified as a target object by reference.
  • the display device 100 can detect not only the direction of itself but also the position, the line of sight before and after the change is assumed on the assumption that the user changes the line of sight with the object of interest in the center of the field of view. It is considered that the object at the intersection of the direction 303 is specified as the target object. However, when the detection accuracy of the orientation and position of the display device 100 is low, the intersection may not be correctly identified.
  • the specifying unit 114 sets an object having a small velocity component that is not caused by the movement of the object as a target object among the in-image speed at which the position where the object is drawn moves in the image generated by the generation unit 113. Therefore, even if the detection accuracy of the orientation and position of the display device 100 is low, the object of interest can be specified.
  • the display device 100 does not identify the target object while the orientation of the display device 100 does not change.
  • the display device 100 may identify the attention object identified last as the attention object for a predetermined period after the last change in the orientation of the display device 100 is detected.
  • the fact that the change in the orientation of the display device 100 is no longer detected is probably because the user has already moved the display device 100 in a direction in which the object of interest can be easily observed.
  • the display device 100 determines the object of interest identified last even if the change of the orientation of the display device 100 is not detected for a predetermined period after the change in the orientation of the display device 100 is last detected. Can be specified as *
  • the specifying unit 114 has the smallest velocity component that is not caused by the movement of the object among the in-image speeds at which the position where the object is drawn in the image generated by the generating unit 113 moves.
  • An example in which is identified as an object of interest has been described.
  • the specifying unit 114 specifies one target object, there is a possibility that the target object is frequently replaced depending on the situation.
  • the display device 100 may not specify another object as the attention object for a predetermined period after the attention object is specified.
  • the change in the orientation of the display device 100 is not limited to this, and for example, a change in roll angle or pitch angle may be detected.
  • an existing personal computer, an information terminal device, or the like can be provided as a display device 100 according to the present invention by applying a program, as well as being provided as a display device 100 having a configuration for realizing the functions according to the present invention. It can also function. That is, an existing personal computer or information terminal device so that a program for realizing each functional configuration by the display device 100 exemplified in the above embodiment can be executed by a CPU or the like that controls the existing personal computer or information terminal device. By applying to, the personal computer and the information terminal device can be functioned as the display device 100 according to the present invention. In addition, the display method according to the present invention can be implemented using the display device 100.
  • Such a program is not limited to the above, and can be applied by an arbitrary method.
  • the program can be stored and applied to a computer-readable recording medium [CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), MO (Magneto Optical Disc), etc.], the Internet, etc. It is also possible to apply the program by storing it in a storage on the network and downloading it.
  • a display device that displays an image, a display method, a program, and a non-transitory computer-readable information recording medium.

Abstract

A detection unit (111) detects changes in orientation of a display device (100) in real space. In accordance with the detected changes in orientation, a change unit (112) changes a line of sight in virtual space where a plurality of objects are standing still or moving. A generation unit (113) generates an image representing a view of the virtual space as seen along the line of sight. A display unit (115) displays the generated image on a display (104) of the display device (100). An identification unit (114) identifies a watched object from among a plurality of objects that are moving at respective speeds within the image, wherein the watched object is an object that exhibits, within the image, a movement speed component which is not attributed to the movement of the object within the virtual space, and which is small as determined in accordance with predetermined criteria.

Description

表示装置、表示方法、プログラム、ならびに、非一時的なコンピュータ読取可能な情報記録媒体Display device, display method, program, and non-transitory computer-readable information recording medium
　本発明は、表示装置、表示方法、プログラム、ならびに、非一時的なコンピュータ読取可能な情報記録媒体に関する。 The present invention relates to a display device, a display method, a program, and a non-transitory computer-readable information recording medium.
　従来から、障害物に隠れてユーザの位置からでは視認が困難な領域やオブジェクトを表示する方法として、例えば特許文献１は、運転者の死角領域をカメラで撮像し、撮像した画像を運転者視点からみた画像に変換するとともに、運転者視点から見た車両の半透明画像を重ね合わせて表示する方法を開示している。 Conventionally, as a method of displaying an area or an object that is hidden behind an obstacle and is difficult to visually recognize from a user's position, for example, Patent Document 1 captures a driver's blind spot area with a camera and captures the captured image as a driver's viewpoint. A method is disclosed in which a translucent image of a vehicle viewed from the driver's viewpoint is superimposed and displayed.
特開２０１０－１１４６１８号公報JP 2010-114618 A
　一方、仮想現実（Virtual Reality）システムや、拡張現実（Augmented Reality）システム、代替現実システムにおいて、ヘッドマウントディスプレイを装着したユーザの向きを検知し、検知された向きに応じて、３次元空間に対する視線方向を変化させ、３次元空間を視線方向に観察した様子を表す画像を生成してヘッドマウントディスプレイに表示する技術がある。特許文献１の技術では、運転者の死角領域を運転者が注目する領域であると予め想定した上で、死角領域の画像を車両の半透明画像と重ね合わせて表示しているが、ヘッドマウントディスプレイに表示する一般的なコンテンツにおいては、ユーザが注目するオブジェクトが予め定められていない場合が多い。このような場合においても、ユーザが注目するオブジェクトを特定する技術が求められる。 On the other hand, in the virtual reality system, augmented reality system, and alternative reality system, the direction of the user wearing the head mounted display is detected, and the line of sight to the three-dimensional space is detected according to the detected orientation. There is a technique for changing the direction and generating an image representing a state in which the three-dimensional space is observed in the line-of-sight direction and displaying the image on the head mounted display. In the technique of Patent Document 1, the blind spot area of the driver is assumed to be an area that the driver pays attention in advance, and the image of the blind spot area is displayed superimposed on the translucent image of the vehicle. In general content displayed on a display, there are many cases where an object to which a user pays attention is not predetermined. Even in such a case, there is a need for a technique for identifying an object that is noticed by the user.
　本発明は、上記の課題を解決するためのものであり、ユーザが注目するオブジェクトを特定可能な表示装置、表示方法、プログラム、ならびに、非一時的なコンピュータ読取可能な情報記録媒体を提供することを目的とする。 The present invention is to solve the above-described problems, and provides a display device, a display method, a program, and a non-transitory computer-readable information recording medium that can identify an object that a user pays attention to. With the goal.
　本発明の第１の観点に係る表示装置は、
　表示装置であって、
　前記表示装置の現実空間における向きの変化を検知する検知部と、
　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化部と、
　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成部と、
　前記生成された画像を、前記表示装置が有する画面に表示する表示部と、
　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定部と、
　を備えることを特徴とする。 A display device according to a first aspect of the present invention provides:
A display device,
A detection unit for detecting a change in orientation of the display device in real space;
A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction;
A display unit for displaying the generated image on a screen of the display device;
Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest A specific part to be
It is characterized by providing.
　上記の観点に係る表示装置において、
　前記複数のオブジェクトの各オブジェクトについて、
　前記視線方向が変化される直前に当該各オブジェクトが前記仮想空間内において存在していた位置の
　　前記視線方向が変化される直前に生成された前記画像内における直前位置と、
　　前記視線方向が変化された直後に生成されるべき前記画像内における直後位置と、
　を求め、
　前記直後位置と前記直前位置との差を、当該各オブジェクトの前記速度成分としてもよい。 In the display device according to the above aspect,
For each object of the plurality of objects,
The immediately preceding position in the image generated immediately before the change of the line-of-sight direction of the position where each object exists in the virtual space immediately before the line-of-sight direction is changed;
A position immediately after in the image to be generated immediately after the line-of-sight direction is changed;
Seeking
A difference between the immediately following position and the immediately preceding position may be used as the velocity component of each object.
　上記の観点に係る表示装置において、
　前記速度成分が最小であるオブジェクトは、前記所定基準で小さくてもよい。 In the display device according to the above aspect,
The object having the minimum velocity component may be small on the predetermined criterion.
　上記の観点に係る表示装置において、
　前記速度成分が所定閾値以下であるオブジェクトは、前記所定基準で小さくてもよい。 In the display device according to the above aspect,
An object whose speed component is equal to or less than a predetermined threshold may be small according to the predetermined reference.
　上記の観点に係る表示装置において、
　前記速度成分が所定閾値未満であるオブジェクトは、前記所定基準で小さくてもよい。 In the display device according to the above aspect,
An object whose velocity component is less than a predetermined threshold may be small according to the predetermined reference.
　上記の観点に係る表示装置において、
　前記検知された向きの変化に対する前記速度成分の比が、所定閾値未満であるオブジェクトは、前記所定基準で小さくてもよい。 In the display device according to the above aspect,
An object whose ratio of the velocity component to the detected change in orientation is less than a predetermined threshold may be small on the predetermined basis.
　上記の観点に係る表示装置において、
　前記変化部は、前記視線方向の変化に応じて、前記注目オブジェクトのうち前記速度成分が最小のオブジェクトを中心に、当該視線方向が変化した方向と逆方向に観察位置を変化させ、
　前記生成部は、前記仮想空間を前記観察位置から前記視線方向に観察した様子を表す画像を生成してもよい。 In the display device according to the above aspect,
The changing unit changes an observation position in a direction opposite to a direction in which the line-of-sight direction has changed, centering on an object having the smallest velocity component among the objects of interest in accordance with a change in the line-of-sight direction,
The generation unit may generate an image representing a state in which the virtual space is observed in the viewing direction from the observation position.
　上記の観点に係る表示装置において、
　前記変化部は、前記視線方向の変化量に比例して、前記観察位置を変化させてもよい。 In the display device according to the above aspect,
The change unit may change the observation position in proportion to a change amount in the line-of-sight direction.
　上記の観点に係る表示装置において、
　前記検知部は、さらに、前記表示装置の位置を検知し、
　前記変化部は、さらに、前記検知された位置に応じて、前記仮想空間における観察位置を変化させ、
　前記生成部は、前記仮想空間を前記観察位置から前記視線方向に観察した様子を表す画像を生成してもよい。 In the display device according to the above aspect,
The detection unit further detects the position of the display device,
The change unit further changes the observation position in the virtual space according to the detected position,
The generation unit may generate an image representing a state in which the virtual space is observed in the viewing direction from the observation position.
　上記の観点に係る表示装置において、
　前記複数のオブジェクトのうち前記注目オブジェクト以外のオブジェクトであって、前記仮想空間を前記視線方向に観察すると前記注目オブジェクトを隠すオブジェクトの透明度を増加させてから、前記画像を生成してもよい。 In the display device according to the above aspect,
The image may be generated after increasing the transparency of an object other than the object of interest among the plurality of objects and hiding the object of interest when the virtual space is observed in the line-of-sight direction.
　本発明の第２の観点に係る表示方法は、
　表示装置が実行する表示方法であって、
　前記表示装置の現実空間における向きの変化を検知する検知ステップと、
　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化ステップと、
　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成ステップと、
　前記生成された画像を、前記表示装置が有する画面に表示する表示ステップと、
　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定ステップと、
　を備えることを特徴とする。 The display method according to the second aspect of the present invention is:
A display method executed by a display device,
A detection step of detecting a change in orientation of the display device in real space;
A change step of changing a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
A generation step of generating an image representing a state in which the virtual space is observed in the line-of-sight direction;
A display step of displaying the generated image on a screen of the display device;
Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest Specific steps to
It is characterized by providing.
　本発明の第３の観点に係るプログラムは、
　コンピュータを、
　表示装置の現実空間における向きの変化を検知する検知部と、
　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化部と、
　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成部と、
　前記生成された画像を、前記表示装置が有する画面に表示する表示部と、
　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定部と、
　として機能させることを特徴とする。 The program according to the third aspect of the present invention is:
Computer
A detection unit that detects a change in orientation of the display device in real space;
A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction;
A display unit for displaying the generated image on a screen of the display device;
Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest A specific part to be
It is made to function as.
　本発明の第４の観点に係る非一時的なコンピュータ読取可能な情報記録媒体は、
　コンピュータを、
　表示装置の現実空間における向きの変化を検知する検知部と、
　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化部と、
　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成部と、
　前記生成された画像を、前記表示装置が有する画面に表示する表示部と、
　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定部と、
　として機能させることを特徴とする。 A non-transitory computer-readable information recording medium according to the fourth aspect of the present invention is provided.
Computer
A detection unit that detects a change in orientation of the display device in real space;
A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction;
A display unit for displaying the generated image on a screen of the display device;
Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest Specific part to do,
It is made to function as.
　なお、上記情報記録媒体は、コンピュータとは独立して配布・販売することができる。ここで、非一時的な（non-transitory）情報記録媒体とは、有形な（tangible）情報記録媒体をいう。非一時的な情報記録媒体は、例えば、コンパクトディスク、フレキシブルディスク、ハードディスク、磁気ディスク、ディジタルビデオディスク、磁気テープ、半導体メモリである。また、一時的な（transitory）情報記録媒体とは、伝送媒体（伝搬信号）それ自体を示す。一時的な記録媒体は、例えば、電気信号、光信号、電磁波である。なお、一時的な（temporary）記憶領域とは、データやプログラムを一時的に記憶するための領域であり、例えばＲＡＭ（Random Access Memory）といった揮発性メモリである。 The above information recording medium can be distributed and sold independently of the computer. Here, the non-transitory information recording medium refers to a tangible information recording medium. Non-temporary information recording media are, for example, compact disks, flexible disks, hard disks, magnetic disks, digital video disks, magnetic tapes, and semiconductor memories. The transitory information recording medium refers to the transmission medium (propagation signal) itself. The temporary recording medium is, for example, an electric signal, an optical signal, or an electromagnetic wave. The temporary storage area is an area for temporarily storing data and programs, and is a volatile memory such as RAM (Random Access Memory), for example.
　本発明によれば、ユーザが注目するオブジェクトを特定することができる。 According to the present invention, it is possible to specify the object that the user pays attention to.
実施形態に係る表示装置のハードウェア構成を示す概略ブロック図である。It is a schematic block diagram which shows the hardware constitutions of the display apparatus which concerns on embodiment. 現実空間において表示装置とユーザの向きが変化する前の様子を示す上面図である。It is a top view which shows a mode before the direction of a display apparatus and a user changes in real space. 現実空間において表示装置とユーザの向きが変化した後の様子を示す上面図である。It is a top view which shows a mode after the direction of a display apparatus and a user changes in real space. 現実空間において表示装置とユーザの向きが変化する前の、仮想空間の様子を示す上面図である。It is a top view which shows the mode of virtual space before the direction of a display apparatus and a user changes in real space. 現実空間において表示装置とユーザの向きが変化した後の、仮想空間の様子を示す上面図である。It is a top view which shows the mode of virtual space after the direction of a display apparatus and a user changes in real space. 実施形態に係る表示装置の機能構成を示す概略ブロック図である。It is a schematic block diagram which shows the function structure of the display apparatus which concerns on embodiment. 現実空間において表示装置とユーザの向きが変化する前に仮想空間を観察位置から視線方向に観察した様子を表す画像である。It is an image showing a state in which the virtual space is observed in the visual line direction from the observation position before the orientation of the display device and the user changes in the real space. 現実空間において表示装置とユーザの向きが変化した後に仮想空間を観察位置から視線方向に観察した様子を表す画像である。It is an image showing a state in which the virtual space is observed in the line-of-sight direction from the observation position after the orientations of the display device and the user change in the real space. 視線方向の変化に起因する速度成分を算出する方法を説明するための図である。It is a figure for demonstrating the method of calculating the speed component resulting from the change of a gaze direction. 現実空間において表示装置とユーザの向きが変化した後に仮想空間を観察位置から視線方向に観察した様子を表す画像である。It is an image showing a state in which the virtual space is observed in the line-of-sight direction from the observation position after the orientations of the display device and the user change in the real space. 実施形態に係る表示装置の制御部により実行される表示処理の流れを表すフローチャートである。It is a flowchart showing the flow of the display process performed by the control part of the display apparatus which concerns on embodiment. 変形例に係る現実空間において表示装置とユーザの向きが変化した後の、仮想空間の様子を示す上面図である。It is a top view which shows the mode of virtual space after the direction of a display apparatus and a user changes in the real space which concerns on a modification. 別の変形例に係る現実空間において表示装置とユーザの向き及び位置が変化した後の様子を示す上面図である。It is a top view which shows a mode after a display apparatus and a user's direction and position changed in the real space which concerns on another modification. 別の変形例に係る現実空間において表示装置とユーザの向き及び位置が変化した後の、仮想空間の様子を示す上面図である。It is a top view which shows the mode of virtual space after the direction and position of a display apparatus and a user change in the real space which concerns on another modification.
　以下に本発明の実施形態を説明する。なお、本実施形態は説明のためのものであり、本願発明の範囲を制限するものではない。従って、当業者であればこれらの各要素若しくは全要素をこれと均等なものに置換した実施形態を採用することが可能であるが、これらの実施形態も本発明の範囲に含まれる。また、本願発明の実施形態を図面を参照して説明するにあたり、図中同一又は相当する部分には同一符号を付す。 Embodiments of the present invention will be described below. In addition, this embodiment is for description and does not limit the scope of the present invention. Accordingly, those skilled in the art can employ embodiments in which each or all of these elements are replaced with equivalent ones, and these embodiments are also included in the scope of the present invention. Further, in describing an embodiment of the present invention with reference to the drawings, the same or corresponding parts in the drawings are denoted by the same reference numerals.
　図１は、本実施形態に係る表示装置１００のハードウェア構成を示す概略ブロック図である。表示装置１００は、例えば、各種のセンサや制御装置を備えたヘッドマウントディスプレイにより構成される。スマートフォンやタブレットコンピュータ、ファブレット等をアタッチメントに装着することにより、ヘッドマウントディスプレイが構築されてもよい。この場合には、スマートフォン等のコンピュータを上記各部として機能させるためのプログラムを、当該スマートフォン等にて実行させることにより、表示装置１００が実現される。図１に示すように、表示装置１００は、制御部１０１と、ＲＯＭ（Read Only Memory）１０２と、ＲＡＭ１０３と、ディスプレイ１０４と、センサ部１０５と、操作部１０６とから構成され、各部は、バス１０７により接続されている。 FIG. 1 is a schematic block diagram showing a hardware configuration of the display device 100 according to the present embodiment. The display device 100 is configured by, for example, a head mounted display including various sensors and a control device. A head mounted display may be constructed by attaching a smartphone, tablet computer, fablet, or the like to the attachment. In this case, the display device 100 is realized by causing the smartphone or the like to execute a program for causing a computer such as a smartphone to function as each unit. As shown in FIG. 1, the display device 100 includes a control unit 101, a ROM (Read Only Memory) 102, a RAM 103, a display 104, a sensor unit 105, and an operation unit 106. 107 is connected.
　制御部１０１は、例えばＣＰＵ（Central Processing Unit）から構成され、表示装置１００全体を制御する。 The control unit 101 includes, for example, a CPU (Central Processing Unit) and controls the entire display device 100.
　ＲＯＭ１０２は、制御部１０１が表示装置１００全体を制御するためのプログラムや各種データを格納する不揮発性メモリである。 The ROM 102 is a non-volatile memory that stores programs and various data for the control unit 101 to control the entire display device 100.
　ＲＡＭ１０３は、制御部１０１が生成した情報や、その情報の生成に必要なデータを一時的に格納するための揮発性メモリである。 The RAM 103 is a volatile memory for temporarily storing information generated by the control unit 101 and data necessary for generating the information.
　ディスプレイ１０４は、ＬＣＤ（Liquid Crystal Display）およびバックライト等を備え、制御部１０１による制御の下、例えば、制御部１０１から出力された画像を表示する。 The display 104 includes an LCD (Liquid Crystal Display), a backlight, and the like, and displays an image output from the control unit 101 under the control of the control unit 101, for example.
　センサ部１０５は、姿勢センサや加速度センサを備え、表示装置１００の向きを検知する。 The sensor unit 105 includes an attitude sensor and an acceleration sensor, and detects the orientation of the display device 100.
　操作部１０６は、ボタン、キーボード、タッチパネル等の入力装置から構成される。操作部１０６は、表示装置１００のユーザからの操作入力を受け付け、受け付けた操作入力に対応する信号を制御部１０１に出力する。 The operation unit 106 includes input devices such as buttons, a keyboard, and a touch panel. The operation unit 106 receives an operation input from the user of the display device 100 and outputs a signal corresponding to the received operation input to the control unit 101.
　次に、本実施形態での現実空間２０１と、表示装置１００に表示される仮想空間（３次元空間）３０１と、における表示装置１００及びユーザ２０２の位置関係について説明する。図２は、現実空間２０１において表示装置１００とユーザ２０２の向きが変化する前（時刻ｔａ）の様子を示す上面図である。図４は、現実空間２０１において表示装置１００とユーザ２０２の向きが変化する前（時刻ｔａ）の、仮想空間３０１の様子を示す上面図である。また、図３は、現実空間２０１において表示装置１００とユーザ２０２の向きが変化した後（時刻ｔｂ）の様子を示す上面図である。図５は、現実空間２０１において表示装置１００とユーザ２０２の向きが変化した後（時刻ｔｂ）の、仮想空間３０１の様子を示す上面図である。以下、これらの図を参照して説明する。なお、これらの図においては、時刻ｔａから時刻ｔｂまでの時間の経過を表すために、英小文字を適宜付与している。以下の説明では、当該英小文字を適宜省略して説明する。 Next, the positional relationship between the display device 100 and the user 202 in the real space 201 and the virtual space (three-dimensional space) 301 displayed on the display device 100 will be described. FIG. 2 is a top view illustrating a state before the direction of the display device 100 and the user 202 changes in the real space 201 (time t a ). FIG. 4 is a top view showing a state of the virtual space 301 before the orientations of the display device 100 and the user 202 change in the real space 201 (time t a ). FIG. 3 is a top view showing a state after the orientations of the display device 100 and the user 202 are changed in the real space 201 (time t b ). FIG. 5 is a top view showing a state of the virtual space 301 after the orientations of the display device 100 and the user 202 are changed in the real space 201 (time t b ). Hereinafter, description will be given with reference to these drawings. In these figures, in order to represent the passage of time from time t a to time t b, it is appropriately impart lowercase. In the following description, the lowercase letters are omitted as appropriate.
　図２及び３に示すように、現実空間２０１において、ユーザ２０２は、表示装置１００を眼前に装着している。ユーザ２０２が表示装置１００を装着しているときには、表示装置１００の表示方向２０３は、ユーザ２０２が表示装置１００を見る視線方向と逆向きになる。 2 and 3, in the real space 201, the user 202 wears the display device 100 in front of the eyes. When the user 202 is wearing the display device 100, the display direction 203 of the display device 100 is opposite to the line-of-sight direction in which the user 202 looks at the display device 100.
　図４及び５に示すように、仮想空間３０１において、静止もしくは移動する複数のオブジェクト４０１～４０４が配置されている。なお、本実施形態においては、オブジェクト４０２，４０３は、静止しており、オブジェクト４０１は矢印Ａ方向に、オブジェクト４０４は矢印Ｂ方向に、それぞれ仮想空間３０１内を移動している。オブジェクト４０１～４０４は、コンピュータグラフィックスにより作成されており、仮想空間３０１内の位置や大きさ、形状、オブジェクト４０１，４０４の移動速度は予め設定されているものとする。一例として、表示装置１００がコンピュータグラフィックスにより再現された駅のホームを表示する場合、オブジェクト４０１，４０４は、電車や人等、駅のホームにおいて移動するオブジェクトであり、オブジェクト４０２，４０３は、柱や看板等、駅のホームにおいて静止しているオブジェクトである。 As shown in FIGS. 4 and 5, a plurality of objects 401 to 404 that are stationary or moving are arranged in the virtual space 301. In this embodiment, the objects 402 and 403 are stationary, and the object 401 moves in the virtual space 301 in the direction of arrow A and the object 404 moves in the direction of arrow B, respectively. The objects 401 to 404 are created by computer graphics, and the position, size, shape, and movement speed of the objects 401 and 404 in the virtual space 301 are set in advance. As an example, when the display device 100 displays a station platform reproduced by computer graphics, the objects 401 and 404 are objects that move in the station platform, such as trains and people, and the objects 402 and 403 are columns. An object that is stationary at the station platform, such as a signboard or a signboard.
　なお、現実空間２０１における方位２１１と仮想空間３０１における方位３１１とは、ずれてもよいし、一致させてもよい。本実施形態において、両者はずれている。 Note that the orientation 211 in the real space 201 and the orientation 311 in the virtual space 301 may be shifted or matched. In this embodiment, both are shifted.
　図６は、本発明の実施形態に係る表示装置１００の機能構成を示す概略ブロック図である。図６に示すように、表示装置１００は、検知部１１１と、変化部１１２と、生成部１１３と、特定部１１４と、表示部１１５と、ディスプレイ１０４とを備える。 FIG. 6 is a schematic block diagram showing a functional configuration of the display device 100 according to the embodiment of the present invention. As illustrated in FIG. 6, the display device 100 includes a detection unit 111, a change unit 112, a generation unit 113, a specification unit 114, a display unit 115, and a display 104.
　検知部１１１は、表示装置１００の現実空間２０１における向きの変化を検知する。例えば、検知部１１１は、センサ部１０５による測定結果から、現実空間２０１における基準軸に対する表示装置１００の表示方向２０３の変化を検知する。現実空間２０１における基準軸としては、現実空間２０１における重力方向、または表示装置１００のディスプレイ１０４の上下方向を採用するのが典型的である。 The detection unit 111 detects a change in orientation of the display device 100 in the real space 201. For example, the detection unit 111 detects a change in the display direction 203 of the display device 100 with respect to the reference axis in the real space 201 from the measurement result by the sensor unit 105. As a reference axis in the real space 201, it is typical to adopt the direction of gravity in the real space 201 or the vertical direction of the display 104 of the display device 100.
　本実施形態において、制御部１０１およびセンサ部１０５が協働することにより、検知部１１１として機能する。 In this embodiment, the control unit 101 and the sensor unit 105 cooperate to function as the detection unit 111.
　変化部１１２は、検知部１１１により検知された表示装置１００の向きに応じて、仮想空間３０１に対する視線方向３０３を変化させる。例えば、まず、変化部１１２は、時刻ｔａから時刻ｔｂまでの間の検知部１１１により検知された表示装置１００の表示方向２０３の変化について、現実空間２０１における基準軸周りの回転量θを取得する。具体的には、図２において、ユーザ２０２が頭の向きを変える前は、表示方向２０３ａは、表示装置１００ａからユーザ２０２ａに向かっている。また、図３において、ユーザ２０２が頭の向きを変えた後は、表示方向２０３ｂは、表示装置１００ｂからユーザ２０２ｂに向かっている。表示方向２０３ａと表示方向２０３ｂとがなす角が基準軸周りの回転量θである。回転量θは、いわゆるヨー角に相当する。図３において、ユーザ２０２は、頭を左に回転させている。 The changing unit 112 changes the line-of-sight direction 303 with respect to the virtual space 301 according to the orientation of the display device 100 detected by the detecting unit 111. For example, firstly, the changing unit 112, the change in the display direction 203 of the display device 100 which is detected by the detection unit 111 between the time t a to time t b, the rotation amount θ around the reference axis in the real space 201 get. Specifically, in FIG. 2, before the user 202 changes the head direction, the display direction 203a is from the display device 100a to the user 202a. In FIG. 3, after the user 202 changes the head direction, the display direction 203b is directed from the display device 100b to the user 202b. The angle formed by the display direction 203a and the display direction 203b is the rotation amount θ around the reference axis. The rotation amount θ corresponds to a so-called yaw angle. In FIG. 3, the user 202 has rotated his head to the left.
　時刻ｔａから時刻ｔｂまでの時間間隔は、用途に応じて定めることができる。例えば、この時間間隔は、表示装置１００のディスプレイ１０４のリフレッシュ間隔（例えば、垂直同期周期）とすることができる。また、後述するように、処理は繰り返し行われるので、ある繰り返し単位における時刻ｔｂは、次の繰り返し単位における時刻ｔａとして解釈される。 Time interval from time t a to time t b may be determined depending on the application. For example, this time interval can be a refresh interval (for example, a vertical synchronization period) of the display 104 of the display device 100. Further, as described later, the process is repeated, the time t b at a repeating unit is interpreted as the time t a in the next repeating unit.
　そして、変化部１１２は、視線方向３０３ａを仮想空間３０１における観察位置３０２ａの基準軸周りに回転量θだけ回転させることにより、視線方向３０３ａを視線方向３０３ｂに変化させる。 Then, the changing unit 112 changes the line-of-sight direction 303a to the line-of-sight direction 303b by rotating the line-of-sight direction 303a around the reference axis of the observation position 302a in the virtual space 301 by the rotation amount θ.
　本実施形態において、制御部１０１が、変化部１１２として機能する。 In the present embodiment, the control unit 101 functions as the changing unit 112.
　生成部１１３は、仮想空間３０１を観察位置３０２から視線方向３０３に観察した様子を表す画像を生成する。例えば、生成部１１３は、オブジェクト４０１～４０４の仮想空間３０１内における所定の位置及び形状に基づいて、透視投影等の技術を用いて、観察位置３０２から視線方向３０３に見たときの画像を生成する。 The generation unit 113 generates an image representing a state in which the virtual space 301 is observed from the observation position 302 in the line-of-sight direction 303. For example, the generation unit 113 generates an image when viewed from the observation position 302 in the line-of-sight direction 303 using a technique such as perspective projection based on a predetermined position and shape of the objects 401 to 404 in the virtual space 301. To do.
　特定部１１４は、複数のオブジェクト４０１～４０４が生成部１１３により生成された画像内にて描画される位置が移動する画像内速度のうち、仮想空間３０１内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する。 The specifying unit 114 is a speed component that is not caused by the movement of the object in the virtual space 301 among the in-image speeds at which the positions where the plurality of objects 401 to 404 are drawn in the image generated by the generation unit 113 move. However, an object that is small on a predetermined basis is identified as the object of interest.
　以下、画像内速度と、オブジェクトの移動に起因しない速度成分について、図７Ａ及び７Ｂを用いて説明する。図７Ａは、時刻ｔａにおいて仮想空間３０１を観察位置３０２から視線方向３０３に観察した様子を表す画像５０１ａであり、図７Ｂは、時刻ｔｂにおいて仮想空間３０１を観察位置３０２から視線方向３０３に観察した様子を表す画像５０１ｂを示す。複数のオブジェクト４０１～４０４のうち、仮想空間３０１内において静止しているオブジェクト４０２，４０３では、視線方向３０３が変化すると、その視線方向３０３の変化に起因する速度成分を画像内速度として、生成された画像５０１ａ，５０１ｂ内で描画される位置が移動する。例えば、オブジェクト４０２では、視線方向３０３の変化に起因する速度成分Ｓ２ｘを画像内速度Ｖ２として、生成された画像５０１ａ，５０１ｂ内で描画される位置が移動する。また、仮想空間３０１内において移動しているオブジェクト４０１，４０４では、視線方向３０３が変化すると、その視線方向３０３の変化に起因する速度成分と、自身の移動に起因する速度成分との合成速度を画像内速度として、生成された画像内で描画される位置が移動する。例えば、オブジェクト４０１では、視線方向３０３の変化に起因する速度成分Ｓ１ｘと、自身の移動に起因する速度成分Ａｘとの合成速度を画像内速度Ｖ１として、生成された画像５０１ａ，５０１ｂ内で描画される位置が移動する。一般的に、ユーザは、移動するオブジェクトに注目している場合、そのオブジェクトを視野、すなわち生成された画像内の中心にとらえた状態でそのオブジェクトの移動に伴って視線方向３０３を変化させると考えられる。その結果、ユーザが注目しているオブジェクト、すなわち注目オブジェクトにおいては、視線方向３０３が変化しても、視線方向３０３の変化に起因する速度成分、すなわち画像内速度のうち、注目オブジェクトの移動に起因しない速度成分は、他のオブジェクトよりも小さいと考えられる。従って、本実施形態では、複数のオブジェクトのうち、オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを注目オブジェクトとして特定する。 Hereinafter, the in-image speed and the speed component not caused by the movement of the object will be described with reference to FIGS. 7A and 7B. Figure 7A is an image 501a representing a state of observation of the virtual space 301 from the viewing position 302 in the viewing direction 303 at time t a, FIG. 7B, a virtual space 301 from the viewing position 302 at time t b in the viewing direction 303 An image 501b representing the observed state is shown. Among the plurality of objects 401 to 404, in the objects 402 and 403 that are stationary in the virtual space 301, when the line-of-sight direction 303 changes, the velocity component resulting from the change in the line-of-sight direction 303 is generated as the in-image speed. The positions drawn in the images 501a and 501b move. For example, in the object 402, the position drawn in the generated images 501a and 501b moves with the speed component S2x resulting from the change in the line-of-sight direction 303 as the in-image speed V2. In addition, in the objects 401 and 404 moving in the virtual space 301, when the line-of-sight direction 303 changes, the combined speed of the speed component due to the change in the line-of-sight direction 303 and the speed component due to its own movement is obtained. As the in-image speed, the position to be drawn in the generated image moves. For example, the object 401 is drawn in the generated images 501a and 501b with the combined speed of the speed component S1x resulting from the change in the line-of-sight direction 303 and the speed component Ax resulting from the movement of the object 401 as the in-image speed V1. Move the position. In general, when a user is paying attention to a moving object, the user views the object in the field of view, that is, the center in the generated image, and changes the line-of-sight direction 303 as the object moves. It is done. As a result, in the object that the user is paying attention to, that is, the attention object, even if the line-of-sight direction 303 changes, the speed component resulting from the change in the line-of-sight direction 303, that is, the in-image speed is caused by the movement of the object of interest. The velocity component that is not considered is considered to be smaller than other objects. Accordingly, in the present embodiment, an object having a small velocity component that does not result from the movement of the object on the basis of a predetermined criterion is identified as the object of interest.
　具体的には、まず、特定部１１４は、各オブジェクト４０１～４０４について、視線方向３０３が変化される直前に各オブジェクト４０１～４０４が仮想空間３０１内において存在していた位置の、視線方向３０３が変化される直前に生成された画像内における直前位置と、視線方向３０３が変化される直後に生成されるべき画像内における直後位置とを求める。具体的には、特定部１１４は、直前位置を、仮想空間３０１内におけるオブジェクト４０１～４０４の既知の位置や移動速度、観察位置３０２及び視線方向３０３から算出する。また、特定部１１４は、直後位置を、直前位置、変化後の観察位置３０２、及び変化後の視線方向３０３から算出する。そして、特定部１１４は、直後位置と直前位置との差を、視線方向３０３の変化に起因する速度成分、すなわちオブジェクトの移動に起因しない速度成分として算出する。なお、特定部１１４は、オブジェクト４０１～４０４の代表点（例えば、中心点や重心点）の位置を、オブジェクト４０１～４０４の位置とする。 Specifically, the identifying unit 114 first sets the line-of-sight direction 303 at the position where each object 401 to 404 existed in the virtual space 301 immediately before the line-of-sight direction 303 is changed. The immediately preceding position in the image generated immediately before the change and the immediately following position in the image to be generated immediately after the line-of-sight direction 303 is changed are obtained. Specifically, the specifying unit 114 calculates the immediately preceding position from the known positions and moving speeds of the objects 401 to 404 in the virtual space 301, the observation position 302, and the line-of-sight direction 303. Further, the specifying unit 114 calculates the immediately following position from the immediately preceding position, the observation position 302 after the change, and the line-of-sight direction 303 after the change. Then, the specifying unit 114 calculates the difference between the immediately following position and the immediately preceding position as a velocity component caused by a change in the line-of-sight direction 303, that is, a velocity component not caused by the movement of the object. The specifying unit 114 sets the positions of the representative points (for example, the center point and the center of gravity) of the objects 401 to 404 as the positions of the objects 401 to 404.
　以下、オブジェクト４０１を例として、図８を用いて視線方向３０３の変化に起因する速度成分を算出する方法について説明する。例えば、図４に示すように時刻ｔａにおいて、オブジェクト４０１がＰａに位置しているとする。このとき、まず特定部１１４は、Ｐａ、観察位置３０２及び視線方向３０３に基づいて、時刻ｔａにおいて生成された画像５０１ａ内におけるＰａの位置Ｐｉａを算出する。そして、特定部１１４は、Ｐａ、時刻ｔｂにおける観察位置３０２、及び時刻ｔｂにおける視線方向３０３に基づいて、時刻ｔｂにおいて生成されるべき画像５０１ｂ内におけるＰａの位置Ｐｉｂを算出する。そして、特定部１１４は、ＰｉｂとＰｉａとの差を、視線方向３０３の変化に起因する速度成分Ｓ１ｘとして算出する。すなわち、特定部１１４は、オブジェクト４０１が静止していると仮定したときのＰｉｂとＰｉａとの差、すなわち画像内速度を視線方向３０３の変化に起因する速度成分Ｓ１ｘとして算出する。特定部１１４は、このようにして仮想空間３０１内のオブジェクト４０１～４０４それぞれについて、視線方向３０３の変化に起因する速度成分を算出する。 Hereinafter, a method for calculating a velocity component caused by a change in the line-of-sight direction 303 will be described using the object 401 as an example with reference to FIG. For example, at time t a as shown in FIG. 4, the object 401 is assumed to be positioned in Pa. In this case, first, a specific unit 114, Pa, based on the observation position 302 and the line-of-sight direction 303, calculates the position Pia in Pa in the image 501a generated at time t a. Then, the specification unit 114, Pa, observation position at time t b 302, and based on the line-of-sight direction 303 at time t b, calculates the position Pib in Pa in the image 501b to be generated at time t b. And the specific | specification part 114 calculates the difference of Pib and Pia as the speed component S1x resulting from the change of the gaze direction 303. FIG. That is, the specifying unit 114 calculates the difference between Pib and Pia when it is assumed that the object 401 is stationary, that is, the in-image speed as the speed component S1x caused by the change in the line-of-sight direction 303. In this way, the specifying unit 114 calculates the velocity component due to the change in the line-of-sight direction 303 for each of the objects 401 to 404 in the virtual space 301.
　そして、特定部１１４は、オブジェクトの移動に起因しない速度成分が、所定基準で小さいか否かを判定する。ここで、オブジェクトの移動に起因しない速度成分が所定基準で小さいか否かを判定する方法の一例として、特定部１１４は、オブジェクト４０１～４０４それぞれについて求められた視線方向３０３の変化に起因する速度成分のうち、最小の速度成分が所定基準で小さいと判定する。そして、特定部１１４は、視線方向３０３の変化に起因する速度成分が最小であるオブジェクト４０１～４０４を注目オブジェクトに特定する。例えば、特定部１１４は、オブジェクト４０１～４０４の視線方向３０３の変化に起因する速度成分のうち、オブジェクト４０１に係る速度成分Ｓ１ｘが最小であるとき、オブジェクト４０１を注目オブジェクトとして特定する。なお、他の例として、視線方向３０３の変化に起因する速度成分が所定閾値以下、または所定閾値未満であるとき、特定部１１４は、その速度成分は所定基準で小さいと判定してもよい。また、特定部１１４は、検知された視線方向３０３の変化に対する速度成分の比が、所定閾値未満であるとき、その速度成分は所定基準で小さいと判定してもよい。具体的には、特定部１１４は、視線方向３０３の変化に起因する速度成分を、検知された視線方向３０３の変化の速度で除した値が、所定閾値未満のとき、その速度成分は所定基準で小さいと判定してもよい。 Then, the specifying unit 114 determines whether or not the speed component that is not caused by the movement of the object is small on a predetermined basis. Here, as an example of a method for determining whether or not the velocity component not caused by the movement of the object is small on a predetermined basis, the specifying unit 114 uses the velocity caused by the change in the line-of-sight direction 303 obtained for each of the objects 401 to 404. Among the components, it is determined that the minimum velocity component is small on a predetermined basis. Then, the specifying unit 114 specifies the objects 401 to 404 having the smallest velocity component due to the change in the line-of-sight direction 303 as the target object. For example, the specifying unit 114 specifies the object 401 as the target object when the speed component S1x related to the object 401 among the speed components resulting from the change in the line-of-sight direction 303 of the objects 401 to 404 is minimum. As another example, when the velocity component resulting from the change in the line-of-sight direction 303 is equal to or less than a predetermined threshold value or less than the predetermined threshold value, the specifying unit 114 may determine that the velocity component is small on a predetermined criterion. Further, when the ratio of the speed component to the detected change in the line-of-sight direction 303 is less than a predetermined threshold, the specifying unit 114 may determine that the speed component is small on a predetermined basis. Specifically, when the value obtained by dividing the speed component due to the change in the line-of-sight direction 303 by the detected speed of change in the line-of-sight direction 303 is less than a predetermined threshold, the specifying unit 114 determines that the speed component is a predetermined reference May be determined to be small.
　本実施形態において、制御部１０１が、特定部１１４として機能する。 In the present embodiment, the control unit 101 functions as the specifying unit 114.
　そして、生成部１１３は、画像を生成する際、オブジェクト４０１～４０４のうち、特定部１１４により特定された注目オブジェクト以外のオブジェクトであって、仮想空間３０１を視線方向３０３に観察すると注目オブジェクトを隠すオブジェクトの透明度を増加させてから、画像を生成する。例えば、生成部１１３は、背景を所定の背景色（例えば黒）で描画し、観察位置３０２から遠いオブジェクトから順に描画する。そして、注目オブジェクト以外の他のオブジェクトを描画すべきピクセルが背景色以外の色で描画されていた場合、生成部１１３は、そのピクセルは、すでに注目オブジェクトの色で描画されている、すなわち当該他のオブジェクトによって隠される注目オブジェクトのピクセルであると判定する。そして、生成部１１３は、当該他のオブジェクトを注目オブジェクトを隠すオブジェクトであると判定し、当該他のオブジェクトの透明度を増加させてから、画像を生成する。例えば、特定部１１４によりオブジェクト４０１が注目オブジェクトとして特定されたとき、生成部１１３は、オブジェクト４０２がオブジェクト４０１を隠すオブジェクトであると判定すると、図９に示すようにオブジェクト４０２の透明度を増加させてから、画像５０１ｂを生成する。生成部１１３は、オブジェクト４０２の透明度を増加させることにより、ユーザが注目する、オブジェクト４０２によって隠されたオブジェクト４０１の部分を視認しやすい画像を生成することができる。なお、注目オブジェクトが他の注目オブジェクトにより隠される場合、生成部１１３は、注目オブジェクトを隠す他の注目オブジェクトの透明度を増加させずに画像を生成する。 When generating the image, the generating unit 113 hides the target object when the virtual space 301 is observed in the line-of-sight direction 303 among the objects 401 to 404 other than the target object specified by the specifying unit 114. Generate the image after increasing the transparency of the object. For example, the generation unit 113 draws the background with a predetermined background color (for example, black), and draws the objects in order from the object far from the observation position 302. When a pixel for drawing an object other than the target object is drawn in a color other than the background color, the generation unit 113 has already drawn the pixel in the color of the target object. It is determined that the pixel of the object of interest is hidden by the object. Then, the generation unit 113 determines that the other object is an object that hides the object of interest, increases the transparency of the other object, and then generates an image. For example, when the identifying unit 114 identifies the object 401 as the object of interest, when the generating unit 113 determines that the object 402 is an object that hides the object 401, the transparency of the object 402 is increased as illustrated in FIG. From this, an image 501b is generated. The generation unit 113 can generate an image that allows the user to pay attention to the part of the object 401 hidden by the object 402 by increasing the transparency of the object 402. Note that when the target object is hidden by another target object, the generation unit 113 generates an image without increasing the transparency of the other target object that hides the target object.
　本実施形態において、制御部１０１が、生成部１１３として機能する。 In the present embodiment, the control unit 101 functions as the generation unit 113.
　表示部１１５は、生成部１１３により生成された画像を、ディスプレイ１０４に表示する。例えば、表示部１１５は、時刻ｔａにおいて図７Ａに示すような画像５０１ａ、時刻ｔｂにおいて図９に示すような画像５０１ｂ、をそれぞれディスプレイ１０４に表示する。 The display unit 115 displays the image generated by the generation unit 113 on the display 104. For example, the display unit 115, the time t a image 501a as shown in FIG. 7A at the time t b image as shown in FIG. 9 at 501b, is displayed on the respective display 104.
　本実施形態において、制御部１０１が、表示部１１５として機能する。 In the present embodiment, the control unit 101 functions as the display unit 115.
　次に、本発明の実施形態にかかる表示装置１００の動作について説明する。図１０は、表示装置１００の制御部１０１により実行される表示処理の流れを表すフローチャートである。本処理は、例えば、操作部１０６を介して本処理の開始指示を受け付けたことを契機として開始する。 Next, the operation of the display device 100 according to the embodiment of the present invention will be described. FIG. 10 is a flowchart showing the flow of display processing executed by the control unit 101 of the display device 100. This process is started, for example, when an instruction to start this process is received via the operation unit 106.
　まず、表示装置１００は、仮想空間３０１内において、観察位置３０２及び視線方向３０３を予め定められた初期状態に設定する（ステップＳ１０１）。 First, the display device 100 sets the observation position 302 and the line-of-sight direction 303 in a predetermined initial state in the virtual space 301 (step S101).
　次に、表示装置１００は、仮想空間３０１を初期化する（ステップＳ１０２）。この初期化の際には、表示装置１００は、例えば、仮想空間３０１内に配置されるオブジェクトの位置、形状、向き、外観を取得、設定したり、仮想空間３０１の無限遠方に配置されると想定される背景の画像等を取得、設定する。 Next, the display device 100 initializes the virtual space 301 (step S102). At the time of this initialization, for example, the display device 100 acquires and sets the position, shape, orientation, and appearance of an object arranged in the virtual space 301, or is arranged at infinity in the virtual space 301. Get and set the assumed background image.
　次に、表示装置１００は、表示装置１００の向きを検知し、ディスプレイ１０４の表示方向が、現実空間２０１において基準軸周りに回転した回転量θを取得する（ステップＳ１０３）。ステップＳ１０３は、繰り返し実行されるので、ここで取得される回転量θは、センサ等により前回測定された表示方向２０３と、今回測定された表示方向２０３と、の差分に相当する。 Next, the display device 100 detects the orientation of the display device 100, and acquires the rotation amount θ that the display direction of the display 104 has rotated about the reference axis in the real space 201 (step S103). Since step S103 is repeatedly executed, the rotation amount θ acquired here corresponds to the difference between the display direction 203 measured last time by a sensor or the like and the display direction 203 measured this time.
　次に、表示装置１００は、取得された回転量θに応じて、視線方向３０３を更新する（ステップＳ１０４）。 Next, the display device 100 updates the line-of-sight direction 303 according to the acquired rotation amount θ (step S104).
　そして、表示装置１００は、仮想空間３０１内のオブジェクト４０１～４０４について、視線方向３０３の変化に起因する速度成分を算出していないオブジェクトを選択する（ステップＳ１０５）。 Then, the display device 100 selects an object for which the velocity component resulting from the change in the line-of-sight direction 303 is not calculated for the objects 401 to 404 in the virtual space 301 (step S105).
　表示装置１００は、ステップＳ１０５において選択されたオブジェクトの直前位置及び直後位置を算出する（ステップＳ１０６）。 The display device 100 calculates the immediately preceding position and the immediately following position of the object selected in Step S105 (Step S106).
　そして、表示装置１００は、ステップＳ１０６において算出された直後位置と直前位置との差を、選択されたオブジェクトの、視線方向３０３の変化に起因する速度成分として算出する（ステップＳ１０７）。そして、表示装置１００は、全てのオブジェクト４０１～４０４について視線方向３０３の変化に起因する速度成分を算出したか否かを判定する（ステップＳ１０８）。表示装置１００は、全てのオブジェクト４０１～４０４について視線方向３０３の変化に起因する速度成分を算出していないとき（ステップＳ１０８；Ｎｏ）、ステップＳ１０５の処理に戻る。 Then, the display device 100 calculates the difference between the immediately after position calculated in step S106 and the immediately preceding position as a velocity component caused by a change in the line-of-sight direction 303 of the selected object (step S107). Then, the display device 100 determines whether or not the velocity component due to the change in the line-of-sight direction 303 has been calculated for all the objects 401 to 404 (step S108). When the display device 100 has not calculated the speed component due to the change in the line-of-sight direction 303 for all the objects 401 to 404 (step S108; No), the display device 100 returns to the process of step S105.
　表示装置１００は、全てのオブジェクト４０１～４０４について視線方向３０３の変化に起因する速度成分を算出したとき（ステップＳ１０８；Ｙｅｓ）算出された速度成分のうち、最小の速度成分に係るオブジェクトを注目オブジェクトとして特定する（ステップＳ１０９）。 When the display device 100 calculates the velocity component due to the change in the line-of-sight direction 303 for all the objects 401 to 404 (step S108; Yes), the object related to the minimum velocity component among the calculated velocity components is selected as the target object. (Step S109).
　表示装置１００は、ステップＳ１０９において特定された注目オブジェクトを隠すオブジェクトがあるか否かを判定する（ステップＳ１１０）。表示装置１００は、注目オブジェクトを隠すオブジェクトがないとき（ステップＳ１１０；Ｎｏ）、ステップＳ１１２の処理に進む。 The display device 100 determines whether there is an object that hides the target object identified in step S109 (step S110). When there is no object that hides the object of interest (step S110; No), the display device 100 proceeds to the process of step S112.
　注目オブジェクトを隠すオブジェクトがあるとき（ステップＳ１１０；Ｙｅｓ）、表示装置１００は、注目オブジェクトを隠すオブジェクトの透明度を所定の度合だけ増加させる（ステップＳ１１１）。 When there is an object that hides the object of interest (step S110; Yes), the display device 100 increases the transparency of the object that hides the object of interest by a predetermined degree (step S111).
　そして、表示装置１００は、観察位置３０２から視線方向３０３に仮想空間３０１を観察した様子を表す画像を生成する（ステップＳ１１２）。 And the display apparatus 100 produces | generates the image showing a mode that the virtual space 301 was observed in the gaze direction 303 from the observation position 302 (step S112).
　この後、表示装置１００は、ディスプレイ１０４の垂直同期割込が生じるまで待機してから（ステップＳ１１３）、生成された画像をディスプレイ１０４に転送してユーザに提示する（ステップＳ１１４）。 Thereafter, the display device 100 waits until a vertical synchronization interrupt occurs on the display 104 (step S113), and then transfers the generated image to the display 104 and presents it to the user (step S114).
　そして、表示装置１００は、仮想空間３０１の状態を更新する（ステップＳ１１５）。例えば、仮想空間３０１が時間の経過に応じて変化するコンピュータグラフィックスにより構成される場合には、オブジェクトの位置や向きを、当該オブジェクトに設定された速度、加速度、角速度、角加速度等により更新する物理シミュレーションを行ったり、予め定められた条件に基づいてオブジェクトを変形させる。 And the display apparatus 100 updates the state of the virtual space 301 (step S115). For example, when the virtual space 301 is configured by computer graphics that changes over time, the position and orientation of the object are updated with the speed, acceleration, angular velocity, angular acceleration, and the like set for the object. A physical simulation is performed or the object is deformed based on a predetermined condition.
　そして、表示装置１００は、ステップＳ１０３の処理に戻る。表示装置１００は、例えば操作部１０６を介して本処理の終了指示を受け付けるまで、繰り返し上記の処理を実行する。なおステップＳ１１３において待機することから、この処理の繰り返し周期は、垂直同期周期となる。 And the display apparatus 100 returns to the process of step S103. The display device 100 repeatedly executes the above-described process until an end instruction for the process is received via the operation unit 106, for example. In addition, since it waits in step S113, the repetition period of this process becomes a vertical synchronizing period.
　以上説明したように、本発明の実施形態に係る表示装置１００は、表示装置１００の現実空間２０１における向きが変化すると、仮想空間３０１における視線方向３０３を変化させ、仮想空間３０１内のオブジェクト４０１～４０４の画像内速度のうち、オブジェクト４０１～４０４の移動に起因しない速度成分が所定基準で小さいオブジェクトを注目オブジェクトに特定する。そのため、ユーザが仮想空間３０１内の移動するオブジェクトを目で追っているとき、表示装置１００は自身の位置を検知できなくても、自身の向きの変化を検知することにより、ユーザが注目するオブジェクトを特定することができる。 As described above, the display device 100 according to the embodiment of the present invention changes the line-of-sight direction 303 in the virtual space 301 when the orientation of the display device 100 in the real space 201 changes, and the objects 401 to 300 in the virtual space 301 change. Among the in-image speeds 404, an object having a small speed component that is not caused by the movement of the objects 401 to 404 is specified as a target object. Therefore, when the user is following the moving object in the virtual space 301, even if the display device 100 cannot detect its own position, the display device 100 can detect the change of its own direction, thereby detecting the object that the user is interested in. Can be identified.
　また、表示装置１００は、特定された注目オブジェクトが他のオブジェクトによって隠されている場合、注目オブジェクトを隠すオブジェクトの透明度を増加させてから、画像を生成する。従って、表示装置１００は、他のオブジェクトによって隠されている注目オブジェクトの部分を視認可能にすることができる。 In addition, when the identified object of interest is hidden by another object, the display device 100 generates an image after increasing the transparency of the object that hides the object of interest. Therefore, the display device 100 can make the visible object portion hidden by other objects visible.
　以上に本発明の実施形態について説明したが、上記実施形態は一例であり、本発明の適用範囲はこれに限られない。即ち、本発明の実施形態は種々の応用が可能であり、あらゆる実施の形態が本発明の範囲に含まれる。 Although the embodiment of the present invention has been described above, the above embodiment is an example, and the scope of application of the present invention is not limited to this. That is, the embodiments of the present invention can be applied in various ways, and all the embodiments are included in the scope of the present invention.
　例えば、上記の実施形態において、表示装置１００は、視線方向３０３の変化に応じて観察位置３０２を変化させることにより、注目オブジェクトののぞき込みを実現してもよい。例えば、図１１に示すように、観察位置３０２ａから視線方向３０３ａに注目オブジェクト４０１ａを見ると、注目オブジェクト４０１ａの一部がオブジェクト４０４ａにより隠されるとする。このとき、ユーザは、オブジェクト４０４ａにより隠されている注目オブジェクト４０１ａの部分を覗き込もうとして、現実空間２０１において頭を回転させるとともに、頭の位置を移動させると想定される。ただし、表示装置１００が自身の位置を検知できないときには、ユーザが頭を移動させても、その移動を検知することができない。このような状況において、表示装置１００は、頭の回転、すなわち表示装置１００の向きの変化を検知し、その向きの変化に応じて仮想空間３０１における視線方向３０３ａを変化させるとともに、視線方向３０３ａの変化に応じて観察位置３０２ａを変化させてもよい。 For example, in the above-described embodiment, the display device 100 may realize the peeping of the object of interest by changing the observation position 302 according to the change in the line-of-sight direction 303. For example, as shown in FIG. 11, when the attention object 401a is viewed from the observation position 302a in the line-of-sight direction 303a, a part of the attention object 401a is hidden by the object 404a. At this time, it is assumed that the user rotates the head and moves the position of the head in the real space 201 in order to look into the portion of the attention object 401 a hidden by the object 404 a. However, when the display device 100 cannot detect its own position, the movement cannot be detected even if the user moves his / her head. In such a situation, the display device 100 detects the rotation of the head, that is, the change in the direction of the display device 100, changes the line-of-sight direction 303a in the virtual space 301 according to the change in the direction, and changes the direction of the line-of-sight direction 303a. The observation position 302a may be changed according to the change.
　具体的には、上記の実施形態において、変化部１１２は、変化させた視線方向３０３の変化に応じて、当該視線方向３０３が変化した方向と逆方向に観察位置３０２を変化させてもよい。例えば、時刻ｔａから時刻ｔｂまでの間に、図２に示す状態から図３に示すようにユーザが頭を反時計回りにθ回転させたとき、変化部１１２は、検知部１１１により検知された表示装置１００の表示方向２０３の変化について、現実空間２０１における基準軸周りの回転量θを取得する。そして、図１１に示すように、変化部１１２は、視線方向３０３ａを仮想空間３０１における観察位置３０２ａの基準軸周りに回転量θだけ回転させることにより、視線方向３０３ａを視線方向３０３ｂに変化させる。また、変化部１１２は、特定部１１４により特定された注目オブジェクト４０１ａの位置Ｐａを中心に、観察位置３０２ａを回転量θだけ視線方向３０３が変化した方向と逆方向に回転移動させることにより、観察位置３０２ａを観察位置３０２ｂに変化させる。図１１に示す例では、視線方向３０３ａは、反時計回りに回転することにより、視線方向３０３ｂに変化しているため、変化部１１２は、観察位置３０２ａを時計回りに回転量θだけ回転移動させることにより、観察位置３０２ｂに変化させる。これにより、表示装置１００が表示装置１００の位置を検出する機能を持たない場合であっても、変化させた視線方向３０３に応じて、視線方向３０３が変化した方向と逆方向に観察位置３０２を変化させることにより、ユーザ２０２の違和感を抑制しつつ観察位置３０２を注目オブジェクトののぞき込みに適切な位置に変化させることができる。 Specifically, in the above-described embodiment, the changing unit 112 may change the observation position 302 in a direction opposite to the direction in which the line-of-sight direction 303 is changed in accordance with the change in the changed line-of-sight direction 303. For example detection, during the period from the time t a to time t b, when a user, as shown in FIG. 3 is rotated θ counterclockwise head from the state shown in FIG. 2, the changing unit 112, the detection unit 111 For the change in the display direction 203 of the display device 100, the rotation amount θ around the reference axis in the real space 201 is acquired. Then, as illustrated in FIG. 11, the changing unit 112 changes the line-of-sight direction 303a to the line-of-sight direction 303b by rotating the line-of-sight direction 303a by the rotation amount θ around the reference axis of the observation position 302a in the virtual space 301. Further, the changing unit 112 rotates the observation position 302a about the position Pa of the target object 401a specified by the specifying unit 114 in the direction opposite to the direction in which the line-of-sight direction 303 has changed by the rotation amount θ, thereby observing. The position 302a is changed to the observation position 302b. In the example shown in FIG. 11, the line-of-sight direction 303a changes to the line-of-sight direction 303b by rotating counterclockwise, and therefore the changing unit 112 rotates the observation position 302a clockwise by the rotation amount θ. Thus, the observation position 302b is changed. Thereby, even when the display device 100 does not have a function of detecting the position of the display device 100, the observation position 302 is set in the direction opposite to the direction in which the line-of-sight direction 303 is changed according to the changed line-of-sight direction 303. By changing, it is possible to change the observation position 302 to an appropriate position for looking into the object of interest while suppressing the user 202 from feeling uncomfortable.
　なお、複数の注目オブジェクトが特定された場合には、変化部１１２は、複数の注目オブジェクトのうち、自身の移動に起因しない速度成分、すなわち視線方向３０３の変化に起因する速度成分が最小のオブジェクトを中心に観察位置３０２を視線方向３０３が変化した方向と逆方向に変化させればよい。また、変化部１１２が観察位置３０２を視線方向３０３が変化した方向と逆方向に回転させる量は、視線方向３０３の回転量θと同じ量に限られない。例えば、変化部１１２は、視線方向３０３の回転量θに比例する回転量で、観察位置３０２を視線方向３０３が変化した方向と逆方向に変化させてもよい。 When a plurality of attention objects are specified, the changing unit 112 selects an object having a minimum velocity component due to a change in the line-of-sight direction 303 among the plurality of attention objects, ie, a velocity component that does not result from its own movement. The observation position 302 may be changed in the direction opposite to the direction in which the line-of-sight direction 303 is changed. Further, the amount by which the changing unit 112 rotates the observation position 302 in the direction opposite to the direction in which the line-of-sight direction 303 changes is not limited to the same amount as the rotation amount θ in the line-of-sight direction 303. For example, the changing unit 112 may change the observation position 302 in a direction opposite to the direction in which the line-of-sight direction 303 is changed with a rotation amount proportional to the rotation amount θ in the line-of-sight direction 303.
　また、上記の実施形態では、表示装置１００が表示装置１００の向きの変化を検知し、その変化に基づいて、注目オブジェクトを特定する例について説明した。しかし、表示装置１００は、さらに表示装置１００の位置を検知する機能を備えてもよい。この場合、変化部１１２は、検知された表示装置１００の位置に応じて、仮想空間３０１における観察位置３０２を変化させてもよい。 In the above-described embodiment, an example has been described in which the display device 100 detects a change in the orientation of the display device 100 and identifies an object of interest based on the change. However, the display device 100 may further have a function of detecting the position of the display device 100. In this case, the changing unit 112 may change the observation position 302 in the virtual space 301 according to the detected position of the display device 100.
　例えば、図１２に示すように、変化部１１２は、時刻ｔａから時刻ｔｂまでの間の検知部１１１により検知された表示装置１００の位置２０４の変化について、移動量ｄを取得する。具体的には、図１２において、ユーザ２０２が頭の向きを変える前は、表示装置１００の代表点（例えば、中心点や重心点）は、位置２０４ａに位置する。また、図１２において、ユーザ２０２が頭の向きを変えた後は、表示装置１００の代表点は、位置２０４ｂに位置する。従って、表示装置１００の移動量ｄは、位置２０４ａと位置２０４ｂの差分ベクトルである。そして、変化部１１２は、図１３に示すように、仮想空間３０１において観察位置３０２ａを移動量ｄだけ移動させることにより、観察位置３０２ａを観察位置３０２ｂに変化させる。そして、生成部１１３は、仮想空間３０１を観察位置３０２ｂから視線方向３０３ｂに観察した様子を表す画像を生成する。そして、特定部１１４は、上記の実施形態と同様に、生成部１１３に生成される画像内でオブジェクトが描画される位置が移動する画像内速度のうち、オブジェクトの移動に起因しない速度成分が所定基準で小さいオブジェクトを注目オブジェクトに特定する。 For example, as shown in FIG. 12, the changing unit 112, the change in the position 204 of the display device 100 which is detected by the detection unit 111 between the time t a to time t b, obtains the movement amount d. Specifically, in FIG. 12, before the user 202 changes the head direction, the representative point (for example, the center point or the center of gravity point) of the display device 100 is located at the position 204a. In FIG. 12, after the user 202 changes the head direction, the representative point of the display device 100 is located at the position 204b. Therefore, the movement amount d of the display device 100 is a difference vector between the position 204a and the position 204b. Then, as illustrated in FIG. 13, the changing unit 112 changes the observation position 302 a to the observation position 302 b by moving the observation position 302 a by the movement amount d in the virtual space 301. Then, the generation unit 113 generates an image representing a state in which the virtual space 301 is observed from the observation position 302b in the visual line direction 303b. Then, as in the above-described embodiment, the specifying unit 114 has a predetermined speed component that is not caused by the movement of the object among the in-image speeds at which the position where the object is drawn in the image generated by the generating unit 113 moves. A small object is identified as a target object by reference.
　一般的に、表示装置１００が自身の向きだけでなく位置も検知できる場合には、ユーザが注目オブジェクトを視野の中心に入れた状態で視線方向を変化させることを想定して、変化前後の視線方向３０３の交点にあるオブジェクトを注目オブジェクトに特定すると考えられる。しかし、表示装置１００の向きや位置の検知精度が低い場合には、その交点を正しく特定できないおそれがある。一方、特定部１１４は、生成部１１３に生成される画像内でオブジェクトが描画される位置が移動する画像内速度のうち、オブジェクトの移動に起因しない速度成分が所定基準で小さいオブジェクトを注目オブジェクトに特定するため、表示装置１００の向きや位置の検知精度が低い場合であっても、注目オブジェクトを特定することができる。 In general, when the display device 100 can detect not only the direction of itself but also the position, the line of sight before and after the change is assumed on the assumption that the user changes the line of sight with the object of interest in the center of the field of view. It is considered that the object at the intersection of the direction 303 is specified as the target object. However, when the detection accuracy of the orientation and position of the display device 100 is low, the intersection may not be correctly identified. On the other hand, the specifying unit 114 sets an object having a small velocity component that is not caused by the movement of the object as a target object among the in-image speed at which the position where the object is drawn moves in the image generated by the generation unit 113. Therefore, even if the detection accuracy of the orientation and position of the display device 100 is low, the object of interest can be specified.
　また、上記の実施形態では、表示装置１００の向きの変化を検知したとき、注目オブジェクトを特定する例について説明した。すなわち、上記の実施形態では、表示装置１００の向きが変化しない間は、表示装置１００は注目オブジェクトを特定しない。しかし、表示装置１００は、例えば表示装置１００の向きの変化を最後に検知してから所定期間、最後に特定された注目オブジェクトを注目オブジェクトとして特定してもよい。表示装置１００の向きの変化が検知されなくなったことは、ユーザがすでに表示装置１００を注目オブジェクトを観察しやすい向きに移動させたためと考えられる。そのため、表示装置１００は、表示装置１００の向きの変化を最後に検知してから所定期間について、表示装置１００の向きの変化が検知されていなくても、最後に特定された注目オブジェクトを注目オブジェクトとして特定することができる。　 In the above embodiment, the example in which the object of interest is specified when the change in the orientation of the display device 100 is detected has been described. In other words, in the above embodiment, the display device 100 does not identify the target object while the orientation of the display device 100 does not change. However, for example, the display device 100 may identify the attention object identified last as the attention object for a predetermined period after the last change in the orientation of the display device 100 is detected. The fact that the change in the orientation of the display device 100 is no longer detected is probably because the user has already moved the display device 100 in a direction in which the object of interest can be easily observed. For this reason, the display device 100 determines the object of interest identified last even if the change of the orientation of the display device 100 is not detected for a predetermined period after the change in the orientation of the display device 100 is last detected. Can be specified as *
　また、上記の実施形態では、特定部１１４は、生成部１１３に生成される画像内でオブジェクトが描画される位置が移動する画像内速度のうち、オブジェクトの移動に起因しない速度成分が最小のオブジェクトを注目オブジェクトに特定する例について説明した。このように、特定部１１４が１つの注目オブジェクトを特定する場合、状況によっては頻繁に注目オブジェクトが入れ替わる可能性がある。このように注目オブジェクトが頻繁に入れ替わることを防ぐため、例えば表示装置１００は、注目オブジェクトを特定してから所定期間は、他のオブジェクトを注目オブジェクトとして特定しないこととしてもよい。 Further, in the above embodiment, the specifying unit 114 has the smallest velocity component that is not caused by the movement of the object among the in-image speeds at which the position where the object is drawn in the image generated by the generating unit 113 moves. An example in which is identified as an object of interest has been described. Thus, when the specifying unit 114 specifies one target object, there is a possibility that the target object is frequently replaced depending on the situation. In order to prevent the attention object from being frequently changed in this manner, for example, the display device 100 may not specify another object as the attention object for a predetermined period after the attention object is specified.
　また、上記の実施形態において、表示装置１００の向きについて、ヨー角の変化を検知する例について説明した。しかし、表示装置１００の向きの変化はこれに限られず、例えばロール角やピッチ角の変化を検知してもよい。 In the above embodiment, the example in which the change in the yaw angle is detected with respect to the orientation of the display device 100 has been described. However, the change in the orientation of the display device 100 is not limited to this, and for example, a change in roll angle or pitch angle may be detected.
　なお、本発明に係る機能を実現するための構成を予め備えた表示装置１００として提供できることはもとより、プログラムの適用により、既存のパーソナルコンピュータや情報端末機器等を、本発明に係る表示装置１００として機能させることもできる。即ち、上記実施形態で例示した表示装置１００による各機能構成を実現させるためのプログラムを、既存のパーソナルコンピュータや情報端末機器を制御するＣＰＵ等が実行できるように、既存のパーソナルコンピュータや情報端末機器に適用することで、そのパーソナルコンピュータや情報端末機器を本発明に係る表示装置１００として機能させることができる。また、本発明に係る表示方法は、表示装置１００を用いて実施できる。 It should be noted that an existing personal computer, an information terminal device, or the like can be provided as a display device 100 according to the present invention by applying a program, as well as being provided as a display device 100 having a configuration for realizing the functions according to the present invention. It can also function. That is, an existing personal computer or information terminal device so that a program for realizing each functional configuration by the display device 100 exemplified in the above embodiment can be executed by a CPU or the like that controls the existing personal computer or information terminal device. By applying to, the personal computer and the information terminal device can be functioned as the display device 100 according to the present invention. In addition, the display method according to the present invention can be implemented using the display device 100.
　また、このようなプログラムは、上記に限られず、任意の方法で適用可能である。プログラムを、例えば、コンピュータが読取可能な記録媒体［ＣＤ－ＲＯＭ（Compact Disc Read-Only Memory）、ＤＶＤ（Digital Versatile Disc）、ＭＯ（Magneto Optical disc）等］に格納して適用できる他、インターネット等のネットワーク上のストレージにプログラムを格納しておき、これをダウンロードさせることにより適用することもできる。 Further, such a program is not limited to the above, and can be applied by an arbitrary method. For example, the program can be stored and applied to a computer-readable recording medium [CD-ROM (Compact Disc Read-Only Memory), DVD (Digital Versatile Disc), MO (Magneto Optical Disc), etc.], the Internet, etc. It is also possible to apply the program by storing it in a storage on the network and downloading it.
　なお、本発明は、本発明の広義の精神と範囲を逸脱することなく、様々な実施の形態及び変形が可能とされるものである。また、上述した実施の形態は、この発明を説明するためのものであり、本発明の範囲を限定するものではない。即ち、本発明の範囲は、実施の形態ではなく、特許請求の範囲によって示される。そして、特許請求の範囲内及びそれと同等の発明の意義の範囲内で施される様々な変形が、この発明の範囲内とみなされる。 The present invention is capable of various embodiments and modifications without departing from the broad spirit and scope of the present invention. The above-described embodiments are for explaining the present invention and do not limit the scope of the present invention. That is, the scope of the present invention is shown not by the embodiments but by the claims. Various modifications within the scope of the claims and within the scope of the equivalent invention are considered to be within the scope of the present invention.
　本発明によれば、画像を表示する表示装置、表示方法、プログラム、ならびに、非一時的なコンピュータ読取可能な情報記録媒体を提供することができる。 According to the present invention, it is possible to provide a display device that displays an image, a display method, a program, and a non-transitory computer-readable information recording medium.
１００（１００ａ，１００ｂ）　表示装置
１０１　制御部
１０２　ＲＯＭ
１０３　ＲＡＭ
１０４　ディスプレイ
１０５　センサ部
１０６　操作部
１０７　バス
１１１　検知部
１１２　変化部
１１３　生成部
１１４　特定部
１１５　表示部
２０１　現実空間
２０２（２０２ａ，２０２ｂ）　ユーザ
２０３（２０３ａ，２０３ｂ）　表示方向
２０４（２０４ａ，２０４ｂ）　表示装置の位置
２１１　現実空間の方位
３０１　仮想空間
３０２（３０２ａ，３０２ｂ）　観察位置
３０３（３０３ａ，３０３ｂ）　視線方向
３１１　仮想空間の方位
４０１～４０４（４０１ａ～４０４ａ，４０１ｂ～４０４ｂ）　オブジェクト
５０１ａ，５０１ｂ　画像 100 (100a, 100b) Display device 101 Control unit 102 ROM
103 RAM
104 display 105 sensor unit 106 operation unit 107 bus 111 detection unit 112 change unit 113 generation unit 114 identification unit 115 display unit 201 real space 202 (202a, 202b) user 203 (203a, 203b) display direction 204 (204a, 204b) display Device location 211 Real space orientation 301 Virtual space 302 (302a, 302b) Observation location 303 (303a, 303b) Viewing direction 311 Virtual space orientation 401-404 (401a-404a, 401b-404b) Object 501a, 501b Image

Claims (13)

  1. 　表示装置であって、
    　前記表示装置の現実空間における向きの変化を検知する検知部と、
    　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化部と、
    　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成部と、
    　前記生成された画像を、前記表示装置が有する画面に表示する表示部と、
    　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定部と、
    　を備えることを特徴とする表示装置。     A display device,
    A detection unit for detecting a change in orientation of the display device in real space;
    A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
    A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction;
    A display unit for displaying the generated image on a screen of the display device;
    Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest A specific part to be
    A display device comprising:
  2. 　前記複数のオブジェクトの各オブジェクトについて、
    　前記視線方向が変化される直前に当該各オブジェクトが前記仮想空間内において存在していた位置の
    　　前記視線方向が変化される直前に生成された前記画像内における直前位置と、
    　　前記視線方向が変化された直後に生成されるべき前記画像内における直後位置と、
    　を求め、
    　前記直後位置と前記直前位置との差を、当該各オブジェクトの前記速度成分とする、
    　ことを特徴とする請求項１に記載の表示装置。     For each object of the plurality of objects,
    The immediately preceding position in the image generated immediately before the change of the line-of-sight direction of the position where each object exists in the virtual space immediately before the line-of-sight direction is changed;
    A position immediately after in the image to be generated immediately after the line-of-sight direction is changed;
    Seeking
    The difference between the immediately following position and the immediately preceding position is the velocity component of each object.
    The display device according to claim 1.
  3. 　前記速度成分が最小であるオブジェクトは、前記所定基準で小さい、
    　ことを特徴とする請求項１に記載の表示装置。     The object having the smallest velocity component is small on the predetermined criterion.
    The display device according to claim 1.
  4. 　前記速度成分が所定閾値以下であるオブジェクトは、前記所定基準で小さい、
    　ことを特徴とする請求項１に記載の表示装置。     An object whose speed component is less than or equal to a predetermined threshold is small on the predetermined basis
    The display device according to claim 1.
  5. 　前記速度成分が所定閾値未満であるオブジェクトは、前記所定基準で小さい、
    　ことを特徴とする請求項１に記載の表示装置。     An object whose speed component is less than a predetermined threshold is small on the predetermined basis,
    The display device according to claim 1.
  6. 　前記検知された向きの変化に対する前記速度成分の比が、所定閾値未満であるオブジェクトは、前記所定基準で小さい、
    　ことを特徴とする請求項１に記載の表示装置。     An object in which the ratio of the velocity component to the detected change in orientation is less than a predetermined threshold is small on the predetermined basis.
    The display device according to claim 1.
  7. 　前記変化部は、前記視線方向の変化に応じて、前記注目オブジェクトのうち前記速度成分が最小のオブジェクトを中心に、当該視線方向が変化した方向と逆方向に観察位置を変化させ、
    　前記生成部は、前記仮想空間を前記観察位置から前記視線方向に観察した様子を表す画像を生成する、
    　ことを特徴とする請求項１に記載の表示装置。     The changing unit changes an observation position in a direction opposite to a direction in which the line-of-sight direction has changed, centering on an object having the smallest velocity component among the objects of interest in accordance with a change in the line-of-sight direction,
    The generation unit generates an image representing a state in which the virtual space is observed from the observation position in the line-of-sight direction;
    The display device according to claim 1.
  8. 　前記変化部は、前記視線方向の変化量に比例して、前記観察位置を変化させる、
    　ことを特徴とする請求項７に記載の表示装置。     The change unit changes the observation position in proportion to the amount of change in the line-of-sight direction.
    The display device according to claim 7.
  9. 　前記検知部は、さらに、前記表示装置の位置を検知し、
    　前記変化部は、さらに、前記検知された位置に応じて、前記仮想空間における観察位置を変化させ、
    　前記生成部は、前記仮想空間を前記観察位置から前記視線方向に観察した様子を表す画像を生成する、
    　ことを特徴とする請求項１に記載の表示装置。     The detection unit further detects the position of the display device,
    The change unit further changes the observation position in the virtual space according to the detected position,
    The generation unit generates an image representing a state in which the virtual space is observed from the observation position in the line-of-sight direction;
    The display device according to claim 1.
  10. 　前記複数のオブジェクトのうち前記注目オブジェクト以外のオブジェクトであって、前記仮想空間を前記視線方向に観察すると前記注目オブジェクトを隠すオブジェクトの透明度を増加させてから、前記画像を生成する、
    　ことを特徴とする請求項１に記載の表示装置。     Generating the image after increasing the transparency of an object other than the object of interest among the plurality of objects and observing the virtual space in the line-of-sight direction and hiding the object of interest.
    The display device according to claim 1.
  11. 　表示装置が実行する表示方法であって、
    　前記表示装置の現実空間における向きの変化を検知する検知ステップと、
    　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化ステップと、
    　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成ステップと、
    　前記生成された画像を、前記表示装置が有する画面に表示する表示ステップと、
    　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定ステップと、
    　を備えることを特徴とする表示方法。     A display method executed by a display device,
    A detection step of detecting a change in orientation of the display device in real space;
    A change step of changing a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
    A generation step of generating an image representing a state in which the virtual space is observed in the line-of-sight direction;
    A display step of displaying the generated image on a screen of the display device;
    Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest Specific steps to
    A display method comprising:
  12. 　コンピュータを、
    　表示装置の現実空間における向きの変化を検知する検知部と、
    　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化部と、
    　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成部と、
    　前記生成された画像を、前記表示装置が有する画面に表示する表示部と、
    　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定部と、
    　として機能させることを特徴とするプログラム。     Computer
    A detection unit that detects a change in orientation of the display device in real space;
    A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
    A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction;
    A display unit for displaying the generated image on a screen of the display device;
    Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest A specific part to be
    A program characterized by functioning as
  13. 　コンピュータを、
    　表示装置の現実空間における向きの変化を検知する検知部と、
    　複数のオブジェクトが静止もしくは移動する仮想空間における視線方向を、前記検知された向きの変化に応じて変化させる変化部と、
    　前記仮想空間を前記視線方向に観察した様子を表す画像を生成する生成部と、
    　前記生成された画像を、前記表示装置が有する画面に表示する表示部と、
    　前記複数のオブジェクトが前記画像内にて描画される位置が移動する画像内速度のうち、前記仮想空間内における当該オブジェクトの移動に起因しない速度成分が、所定基準で小さいオブジェクトを、注目オブジェクトに特定する特定部と、
    　として機能させることを特徴とするプログラムを記録した非一時的なコンピュータ読取可能な情報記録媒体。     Computer
    A detection unit that detects a change in orientation of the display device in real space;
    A changing unit that changes a line-of-sight direction in a virtual space where a plurality of objects are stationary or moving according to the change in the detected direction;
    A generating unit that generates an image representing a state in which the virtual space is observed in the line-of-sight direction;
    A display unit for displaying the generated image on a screen of the display device;
    Among the in-image velocities at which the positions where the plurality of objects are drawn in the image move, the object whose velocity component that is not caused by the movement of the object in the virtual space is small on a predetermined basis is identified as the object of interest A specific part to be
    A non-transitory computer-readable information recording medium having a program recorded thereon, wherein the program is recorded as a program.
PCT/JP2018/012034 2018-03-26 2018-03-26 Display device, display method, program, and non-transitory computer-readable information recording medium WO2019186622A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2018/012034 WO2019186622A1 (en) 2018-03-26 2018-03-26 Display device, display method, program, and non-transitory computer-readable information recording medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2018/012034 WO2019186622A1 (en) 2018-03-26 2018-03-26 Display device, display method, program, and non-transitory computer-readable information recording medium

Publications (1)

Publication Number Publication Date
WO2019186622A1 true WO2019186622A1 (en) 2019-10-03

Family

ID=68062379

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2018/012034 WO2019186622A1 (en) 2018-03-26 2018-03-26 Display device, display method, program, and non-transitory computer-readable information recording medium

Country Status (1)

Country Link
WO (1) WO2019186622A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0950541A (en) * 1995-08-10 1997-02-18 Sega Enterp Ltd Virtual picture generating device and its method
JP2014044476A (en) * 2012-08-24 2014-03-13 Fujitsu Ltd Image processor and image processing method
JP2017033465A (en) * 2015-08-05 2017-02-09 株式会社コロプラ Method and program

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0950541A (en) * 1995-08-10 1997-02-18 Sega Enterp Ltd Virtual picture generating device and its method
JP2014044476A (en) * 2012-08-24 2014-03-13 Fujitsu Ltd Image processor and image processing method
JP2017033465A (en) * 2015-08-05 2017-02-09 株式会社コロプラ Method and program

Similar Documents

Publication Publication Date Title
CN106502427B (en) Virtual reality system and scene presenting method thereof
CN107920734B (en) Sight line detection method and device
EP3526774B1 (en) Modifying hand occlusion of holograms based on contextual information
CN112384843B (en) Dynamic panel mask
US20210368152A1 (en) Information processing apparatus, information processing method, and program
US11212501B2 (en) Portable device and operation method for tracking user's viewpoint and adjusting viewport
US20190230290A1 (en) Information processing device, information processing method, and program
US10762691B2 (en) Techniques for compensating variable display device latency in image display
CN114026603A (en) Rendering computer-generated reality text
US11410330B2 (en) Methods, devices, and systems for determining field of view and producing augmented reality
US9965697B2 (en) Head pose determination using a camera and a distance determination
WO2021124920A1 (en) Information processing device, information processing method, and recording medium
WO2019186622A1 (en) Display device, display method, program, and non-transitory computer-readable information recording medium
US20220036779A1 (en) Information processing apparatus, information processing method, and recording medium
JP6713591B1 (en) Display control device, display control method, program, and non-transitory computer-readable information recording medium
US9959637B2 (en) Method and apparatus for processing border of computer figure to be merged into background image
US10854010B2 (en) Method and device for processing image, and recording medium
US20210281966A1 (en) Information processing apparatus, program, and information processing method
KR20160128735A (en) Display apparatus and control method thereof
US11887151B2 (en) Method and apparatus for providing advertisement disclosure for identifying advertisements in 3-dimensional space
US11615767B2 (en) Information processing apparatus, information processing method, and recording medium
JPH0955959A (en) Picture display
JP4989427B2 (en) Image data generator
CN117730298A (en) Screen interaction using EOG coordinates
JP2019197409A (en) Image processing apparatus, image processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18912456

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18912456

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP