WO2020059028A1

WO2020059028A1 - Head-mounted display, and object display method

Info

Publication number: WO2020059028A1
Application number: PCT/JP2018/034518
Authority: WO
Inventors: 橋本　康宣; 清水　宏; 貞雄鶴賀; 尚久高見澤; 益岡　信夫; 川前　治; 明石山
Original assignee: マクセル株式会社
Priority date: 2018-09-18
Filing date: 2018-09-18
Publication date: 2020-03-26

Abstract

A detector 71 of a head-mounted display 100 detects the degree of rotation of the head-mounted display 100, and a display controller 72 displays an object in a display reference position that exceeds the angle of rotation on the basis of the degree of rotation detected by the detector 71. Thus, because the object at the display reference position exceeding the angle of rotation is added and displayed, it is possible to efficiently search for the object.

Description

Head mounted display and object display method

The present invention relates to a head-mounted display and an object display method.

In recent years, wearable devices that can be worn on the body and used in a hands-free state have been developed, and a wide variety of products and services are already known. A typical product of the wearable device is a head-mounted display (HMD), which is a display device used by being worn on the head.

The head-mounted display is a see-through type (transmissive type) that allows the real field of view to be visually superimposed on the display image, depending on the display image observation method, and observes only the display image by blocking incident light from the real world. It is classified as a non-see-through type (closed type) that can be used.

In addition, it is possible to execute an immersive game using a conventional head-mounted display, realize a 360 ° panoramic image or virtual space on the head-mounted display by rotating the head, and realize an immersive video expression. Has been done.

In recent years, in addition to these, it has begun to be used for displaying information such as applications. For example, a large number of images (information) are virtually arranged so as to surround the user (hereinafter, referred to as an information space), and the head is rotated from among them, and a visual field image (information) selected from the information space is obtained. ) Is displayed. Thereby, various information can be efficiently displayed by a simple operation of the user such as rotation of the head.

As an example, Patent Literature 1 below detects whether a wearer of a head-mounted display is walking or stopping, and when the wearer is stopped, the information space is linked to the movement of the wearer's head. A technique for displaying a display object (information) selected from within is described.

JP 2016-82411 A

However, the technology described in Patent Document 1 displays an object in the information space in conjunction with the movement of the head, but is not always sufficient in consideration of search efficiency.

An object of the present invention is to provide a head-mounted display capable of efficiently searching for an object.

技術 As a means for solving the above-mentioned problems, the technology described in the claims is used.

As an example, a head-mounted display of the present invention is a head-mounted display that controls display of an object whose display reference position is determined in advance, and a detector that detects a rotation degree of the head-mounted display, and a detector that detects the rotation degree of the head-mounted display. A display controller that displays an object at a display reference position exceeding the rotation angle based on the rotation degree thus performed.

オブジェクト By using the technology of the present invention, objects can be efficiently searched.

It is a figure showing an outline of a head mounted display system concerning this embodiment. FIG. 3 is a diagram illustrating a hardware configuration of a head mounted display. FIG. 2 is a diagram illustrating a hardware configuration of an information processing device. It is a functional block diagram of a head mounted display system. FIG. 4 is an explanatory diagram of an example of an information space. It is explanatory drawing of a cylindrical information space. It is a figure showing the example of the information display displayed on the information space. FIG. 4 is an explanatory diagram of a position in a virtual display space. FIG. 3 is a diagram illustrating an example of correspondence between an information space and a virtual display space. FIG. 6 is a diagram illustrating a graph of a function of λ. It is a diagram illustrating a graph of a function of V _lambda. It is a figure showing the example of the screen which controls fixation and release. FIG. 4 is a diagram illustrating a graph of a function of scroll control. FIG. 4 is a diagram illustrating a graph of a function of scroll control. It is a figure showing the example which divided the virtual display space into a plurality of fields. FIG. 4 is a diagram illustrating a graph that defines a range of an information space to be displayed in a virtual display space. FIG. 3 is an explanatory diagram of a virtual display space displayed on a head mounted display. It is a figure showing the example of an image display which has a plane part and a curved part. FIG. 9 is an explanatory diagram showing an example in which a high-speed search is possible in a virtual display space displayed on a head-mounted display. It is a flowchart which shows the process which displays the information of an information space on a head mounted display. It is a flowchart which shows the process which extracts the display image of the range of the angle of view of the viewer from the information space. 9 is a flowchart illustrating processing of the head mounted display according to the second embodiment. It is a flowchart which shows the process which extracts the display image of the range of the angle of view of the viewer from the information space. It is a flowchart which shows the process which displays the information of an information space on a head mounted display.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiment described below is an example for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the apparatus to which the present invention is applied and various conditions. The present invention is not limited to the following embodiment. Further, a part of each embodiment described later may be appropriately combined and configured.

(Example 1)
FIG. 1 shows a configuration diagram of the head mounted display system of the present embodiment. The head mounted display 100 with the camera 140 is connected to the information processing device 200 via wireless communication, and displays a display image selected from the information space.

Here, the information space is an information space in which objects such as images and application screens are associated with positions (display reference positions) serving as references when the objects are displayed.

(4) Further, a keyboard 300 is connected as input means of the information processing device 200. In the present embodiment, an example of the keyboard 300 is described as an input unit. However, the present invention is not limited to this, and a camera is provided in the information processing apparatus 200, input by a gesture of an operator, and a controller held by the operator in a hand. Input, voice input, and the like.

FIG. 2 is a hardware configuration diagram of the head mounted display 100. As shown in FIG. 2, the head-mounted display 100 includes a controller 101, a memory 102, a display 103, a GPS receiver 104, a voice input unit 105, a voice output unit 106, an input operation unit 107, a posture sensor unit 110, and a communication unit 120. , And a camera 140. When the above-described components operate, the functions of the head-mounted display 100 described later are exhibited.

The controller 101 is a microprocessor unit (arithmetic processing device) that controls the entire head mounted display 100 according to a predetermined program stored in the memory 102.

The memory 102 serves as a program area for executing the basic operation program and other application programs and a temporary storage area for temporarily holding data as needed when executing various application programs. The memory 102 may be integrated with the controller 101. The memory 102 is, for example, a RAM (Random Access Memory) or the like.

The display 103 has a half-mirror and functions as a so-called see-through type head-mounted display that can display an image while transmitting external light.

The GPS (Global Positioning System) receiver 104 receives position information (GPS signals) from GPS satellites.

The audio input unit 105 is a microphone, and converts the voice of the user of the head mounted display 100 and the like into audio data for input. Note that the audio input unit 105 is separate from the head mounted display 100, and may be connected to the head mounted display 100 by wire communication or wireless communication.

The audio output unit 106 is a speaker or the like, and outputs audio data sent from the audio input unit 105 or the information processing device 200 as audio.

The input operation unit 107 inputs an operation instruction to the head mounted display 100. The operation input instruction may be executed via one or both of the input operation device 107 of the head mounted display 100 and the input operation device 205 of the information processing device 200 described later.

The attitude sensor unit 110 is a group of sensors for detecting the state of the head mounted display 100, and includes an acceleration sensor 111, a gyro sensor 112, and a geomagnetic sensor 113.

The acceleration sensor 111 has a function of detecting at least one of the acceleration and the tilt angle of the head mounted display 100. The gyro sensor 112 has a function of detecting the direction of movement of the head-mounted display 100, and acquires gyro information indicating angular velocity data accompanying a change in the direction of the head-mounted display 100. The geomagnetic sensor 113 is a type of azimuth detecting device that detects an angle indicating the absolute azimuth of the head mounted display 100 based on terrestrial magnetism. An example of such an angle is an azimuth angle.

(4) These sensors detect posture information such as the direction in which the head mounted display 100 is facing, the rotation angle, and the inclination. Other sensors may be further provided.

The communication unit 120 includes a LAN (Local Area Network) communication device 121, a telephone network communication device 122, a short-range wireless communication device 123, and a dedicated communication device 124. The LAN communicator 121 transmits and receives data by connecting to a wireless communication access point of the Internet by wireless communication. The telephone network communication unit 122 performs telephone communication (call) and data transmission / reception by wireless communication with a base station of a mobile telephone communication network. The short-range wireless communication device 123 performs wireless communication when approaching a corresponding reader / writer. Each of the LAN communication device 121, the telephone network communication device 122, and the short-range wireless communication device 123 includes an encoding circuit, a decoding circuit, an antenna, and the like. Further, an infrared communication unit or the like may be provided. The dedicated communication device 124 is a dedicated interface for performing communication with the information processing device 200, and performs wired or wireless communication. Further, communication with the information processing device 200 may use another communication device.

The camera 140 captures an image of the external world in the direction of the face and obtains the image as image data. In this embodiment, the head mounted display 100 is described as a see-through type head mounted display, but may be a non-see-through type head mounted display. In this case, an equivalent function can be realized by combining and displaying the image data of the outside world acquired by the camera and the information display image.

FIG. 3 is a hardware configuration diagram of the information processing device 200. As shown in FIG. 3, the information processing device 200 includes a controller 201, a memory 202, a display 203, a storage 204, an input operation device 205, and a communication unit 220. When the above-described components operate, the functions of the information processing device 200 described later are exhibited.

The information processing device 200 is a digital device such as a PC (Personal Computer), a server, and a tablet terminal. Note that the information processing apparatus 200 may be configured to be able to access information such as a cloud via the communication unit 220.

The controller 201 is a microprocessor unit (arithmetic processing device) that controls the entire information processing apparatus 200 according to a predetermined program stored in the storage 204.

The memory 202 serves as a program area for executing the basic operation program and other application programs and a temporary storage area for temporarily holding data as needed when executing various application programs. The memory 202 is, for example, a RAM (Random Access Memory) or the like. The memory 202 may be integrated with the controller 201.

(4) The display 203 displays various display data. The storage 204 stores various setting values such as information of a wearer of the head mounted display 100, images captured by the camera 140, and the like. Further, information space information serving as information display data of an application or the like is stored. Further, it is assumed that the information processing apparatus 200 can be expanded in function by downloading a new application program from the application server via the Internet.

At this time, the downloaded new application program is stored in the storage 204. When the controller 201 expands the new application program stored in the storage 204 into the memory 202 and further executes the expanded new application program, the information processing apparatus 200 can realize various new functions. I do.

The storage 204 needs to hold the stored information even when the power is not supplied to the information processing apparatus 200. Therefore, for example, devices such as a flash ROM, an SSD (Solid State Drive), and an HDD (Hard Disk Drive) are used.

The keyboard 300 is connected to the input operation device 205 as an operation input device. However, the present invention is not particularly limited to the keyboard 300 as long as operation instructions for the head mounted display 100 can be input.

Note that the configuration example of the head mounted display system shown in FIG. 2 includes many configurations that are not essential to the present embodiment, such as a part of the

communication units

120 and 220 and a part of the attitude sensor unit 110. Even if the configuration is not provided, the effect of the present embodiment is not impaired. Further, a configuration (not shown) such as an illuminance sensor and a proximity sensor may be further added.

Next, the functions of the head mounted display 100 and the information processing device 200 will be described using a functional block diagram of the head mounted display system shown in FIG.

The head-mounted display 100 includes a detector 71, a display controller 72, a receiver 73, and a communication unit 74.

The detector 71 is a part that detects the degree of rotation of the head mounted display 100. The detector 71 is realized by the attitude sensor unit 110.

The detector 71 detects any one of a rotation angle, an angular velocity, and an angular acceleration as the degree of rotation of the head mounted display 100. The detector 71 detects the degree of rotation of the head mounted display 100 and sends the detection result to the display controller 72.

The display controller 72 is a part that displays an object at a display reference position exceeding the rotation angle based on the rotation degree detected by the detector 71.

For example, when the rotation degree is equal to or more than a predetermined value, the display controller 72 displays a display reference position object exceeding the rotation angle.

The display controller 72 is realized by the controller 101 and the display 103. The accepting unit 73 is a unit that accepts designation of an object to be displayed by the display controller 72. The reception device 73 is realized by the input operation device 107. In this case, the display controller 72 fixedly displays the object whose specification has been received by the receiver 73.

The communication unit 74 is realized by the communication unit 120, and performs communication with the information processing device 200.

The information processing device 200 includes a memory 81, an object extraction controller 82, and a communication unit 83. The memory 81 is a part for storing various information such as objects. The memory 81 is realized by the memory 202 and the storage 204.

The object extraction controller 82 is a part for extracting an object based on the degree of rotation of the head mounted display 100. The object extraction controller 82 is realized by the controller 201, for example.

The communication unit 83 is realized by the communication unit 220, and performs communication with the head mounted display 100.

The information space described above links the display reference position of the object. FIG. 5A is an explanatory diagram of an example of an information space in which an information display image is virtually arranged around a wearer of the head mounted display 100. Here, the information space is a three-dimensional space, and as shown in FIG. 5 (b), the coordinates in the space are represented by a horizontal rotation angle 0 (clockwise when viewed from above with the user's front being 0 degrees). ), The vertical rotation angle ζ (the horizontal plane is 0 degrees, and the upward direction is +), and the distance R from the center. Each object is linked to coordinates in the information space.

The point where the angle is used for the coordinate value is similar to the polar coordinate, but the line element ds that determines the distance in space is defined by the following equation (Equation 1) and is different from the polar coordinate (here, the angle is expressed in radians. Do).

Further, the point different from the polar coordinates is that the angular coordinates ξ and ζ have no limitation on the values that can be taken from minus infinity to plus infinity. Thereby, infinite information can be arranged in principle. Hereinafter, this type of information space is referred to as a spherical information space.

インフォメーション As another information space, an information space based on cylindrical coordinates can be considered. This information space is called a cylindrical information space.

FIG. 6A is an explanatory diagram of the cylindrical information space. The cylindrical information space uses cylindrical coordinates having a radius R and a height H centered on a vertical line Az. Note that the radius R and the height H are appropriately set for each wearer of the head mounted display 100. In the cylindrical information space where the information display image is virtually arranged, the range of ± 180 ° with respect to the front is displayed in cylindrical coordinates, and the coordinate values beyond that are overlapped in the same cylindrical coordinate system, but the hierarchy is They are identified by different things. That is, since the clockwise circle and the counterclockwise circle overlap and exist in a multiplex manner, in principle, the cylindrical information space can also handle infinite information display data.

Specifically, in the case of clockwise rotation, up to 180 ° represents the cylindrical information space of the first hierarchy, and from 180 ° to less than 540 ° represents the cylindrical information space of the second hierarchy, and thereafter, every 360 ° increase. The hierarchy increases. The same applies to the case of counterclockwise rotation. The field of view V is an area indicated by the height Hv and the circumference Lv set in cylindrical coordinates, and is an area displayed on the display 103 of the head mounted display 100. Therefore, in other words, the cylindrical information space can be said to be a virtual space obtained by folding a huge plane information display space into a cylindrical shape.

FIG. 6B is a schematic diagram in which the visual field V is developed from cylindrical coordinates to planar coordinates. The intersection between the reference direction indicating the front direction initially set by the wearer of the head mounted display 100 and the cylindrical information space is the origin. The field of view V is set within a range of the left and right rotation angles ± θ around the Az axis orthogonal to the reference direction and the vertical elevation angles ± β from the reference direction.

(4) The position of the visual field V in the cylindrical information space is moved by moving the head in the rotation angle or the elevation direction from the reference direction. The movement in the roll angle direction with the reference direction as the rotation axis is ignored. The values of the angle β and the angle θ are appropriately set by the physical specifications of the display 103 of the head mounted display 100 and the wearer.

FIG. 7A is an example of information display displayed in the information space. In the present embodiment, the display screen for each application is arranged in a tile shape, and the size of the field of view V is the same as the size of the display screen of one application. However, the present invention is not limited to this, and a display screen of an application adjacent to the visual field V may be included. The wearer of the head-mounted display 100 can appropriately set and change the display screen including the vertically and horizontally separated display screens according to the purpose of use via the input operation device 107 or 205.

FIG. 7B is another example of information display displayed in the information space. Here, an example of an information display screen of one application is shown. Specifically, it is an information display screen of a spreadsheet application, in which table data is arranged vertically and horizontally. The field of view V corresponds to a partial area of the table data area. Note that the range of the partial area of the table data can be appropriately changed by the wearer through an operation input for enlargement / reduction.

In the following, an embodiment using a spherical information space will be described and simply referred to as an information space.

(4) The information is arranged in the information space, and the information display surface is not limited to a curved surface but may be arranged in a plane. Further, it may be a 3D object.

As shown in FIG. 8, the position of the head-mounted display 100 in the virtual display space is represented by r from the center, the horizontal angle of the direction in which the head-mounted display 100 is directed to λ (assuming the user's front is 0 degrees). The clockwise direction when viewed from above is +), and the angle in the vertical direction is μ (the horizontal plane is 0 ° and the upward direction is +).

The display viewing angle range in the virtual display space of the head mounted display 100 is set to ± [delta] _S in the horizontal direction and the perpendicular direction ± [delta] _P. To display the information in the display information space, the directions (方向, ξ) in the information space are specified from the directions (λ, μ) in which the center of the display field of view in the virtual display space of the head mounted display 100 faces. . Then, the head-mounted display 100 converts information arranged in a horizontal direction ± δ ' _S and a vertical direction ± δ' _{P in} the information space around the designated direction (ξ, ζ). The image is displayed as an image in the virtual display space of the head mounted display 100. Usually, δ ′ _S = δ _S and δ ′ _P = δ _P.

(4) The vertical direction of the information space and the virtual display space remains the same as the reference direction, and is usually adjusted to the vertical direction of the outside world. Therefore, even if the wearer of the head-mounted display 100 makes a movement in which the head is tilted about the line of sight, the displayed information does not rotate and the direction is kept constant with respect to the vertical direction of the outside world. Is displayed.

As a specific correspondence between the direction in the virtual display space of the head mounted display 100 and the direction of the information space, for example, one or more coefficients k _1S and k _1P are set, and the following correspondence is used. I do. FIGS. 9A and 9B show the correspondence only in the horizontal direction (the same applies to the vertical direction).

Taking the horizontal direction as an example, if the coefficient is 1, the object at the reference display position at the same angle (the above angle ξ) as the rotation angle (the above angle λ) is displayed. The object at the reference display position at an angle (the angle ξ) larger than the rotation angle (the angle λ) is displayed. If the coefficient is larger than 1, the rotation speed is increased, and a high-speed search can be performed. Furthermore, since there is no need to set an upper limit on the coordinate value of the information space, it is possible to use a space as large as necessary corresponding to a large coefficient. That is, a large amount of information can be displayed.

For example, the display controller 72 detects the rotation degree (the above-described angle λ) detected by the detector 71, calculates the angle ξ of the information space based on the rotation degree, and transmits the angle ξ through the communication unit 74. To the information processing apparatus 200. The object extraction controller 82 of the information processing device 200 extracts a display image (a set of objects to be displayed) with the angle ξ as a display reference position from the memory 81, and sends the extracted display image to the head mounted display 100. . The display controller 72 displays the display image on the display 103. Note that the display controller 72 transmits the angle λ to the information processing device 200, and the object extraction controller 82 calculates the angle ξ in the information space based on the angle λ, and sets the angle 表示 as the display reference position. A display image to be displayed may be extracted.

Thus, the display controller 72 also displays an object whose display reference position is a position (the angle ξ) exceeding the rotation angle (for example, the angle λ).

Further, it may be displayed in the virtual display space according to the distance R at which the object is arranged in the information space, or may be displayed at a different distance position (meaningful in 3D display). For example a positive coefficient as k _R, set by the following equation.

In this case, by displaying the object in the information space in 1 / k _1R times to display space, the overlapping relationship between objects in the information space can be maintained.

Further, the coefficients k _1S and k _1P need not be constant values, and may be changed according to the orientation of the head mounted display 100 in the virtual display space. For example, a setting may be made such that the coefficient is set to 1 near the front, and to a value larger than 1 when the coefficient exceeds a certain range.

More generally, it is assumed that (ξ, ζ) is determined by the function of (λ, μ) as shown in the following equation 4, and that the slope of the function is 1 or more as shown in the following equation 5 I do. Although only f ₁ (λ) is shown in FIG. 10, the same applies to g ₁ (μ) as shown in the following Expression 6.

Further, the rate of change of (ξ, ζ) with respect to the change of (λ, μ) may be controlled as in the following

equations

7 and 8.

For example, usually, k _2S and k _2P may be set to 1 and their values may be controlled by a user's instruction (wearer's instruction).

As one modified example of the user instruction, the angular acceleration of rotating the head-mounted display 100 is monitored, and when an angular acceleration equal to or more than a certain value is detected, the value of k _2S or k _{2P in} the direction is changed. A method of changing the value to a value larger than 1 may be used. At this time, the value of the coefficient may be automatically returned to 1 on the condition that a certain time elapses or the value of the angular acceleration is equal to or less than the certain value for a certain time.

For example, the detector 71 detects the angular acceleration and sends the detected angular acceleration to the display controller 72. When the angular acceleration is equal to or greater than the predetermined threshold, the display controller 72 sets the value of k _2S or k _{2P to} a value greater than 1, and sets the angular acceleration based on the value to the information processing device 200. May be sent to In this case, the object extraction controller 82 of the information processing device 200 extracts a display image based on the angular acceleration.

Further, the values of the coefficients k _{2S and} k _2P may be controlled according to the rotational angular velocity of the head mounted display 100. For example, when the rotational angular velocity is below a certain value, the values of k _2S and k _2P are set to 1, and when the rotational angular speed is above a certain value, the values of k _2S and k _2P are set to values larger than 1. Alternatively, generally, k _2S and k _2P may be set as functions of the rotational angular velocity as in the following Expressions 9 and 10, and may be one or more values.

Here, V _λ and V _μ are the magnitudes of the rotational angular velocities in the horizontal and vertical directions, respectively, as shown in the following Expressions 11 and 12. In order to suppress the influence of the head swing, a value obtained by reducing the high frequency component of the rotational angular velocity may be used.

For example, the detector 71 detects the rotational angular velocity and sends the detected rotational angular velocity to the display controller 72. The display controller 72 sets the value of k _2S or k _{2P to} a value greater than 1 when the rotation angular speed is equal to or greater than a predetermined threshold, and sets the rotation angular speed based on the value to the information processing device 200. May be sent to In this case, the object extraction controller 82 of the information processing device 200 extracts a display image based on the rotation angular velocity.

FIG. 11 shows an example of a function only in the horizontal direction. The same applies to the vertical direction. In addition to the control of the change rate, an operation of temporarily fixing a specific direction of the information space to a viewer of the virtual display space of the head mounted display 100 may be performed. Specifically, on the viewer screen, the accepting unit 73 accepts a specification for a specific object in the information space. In response to this, the display controller 72 fixedly displays the object whose specification has been received by the receiver 73.

ヘッド With the object fixedly displayed as described above, the head mounted display 100 is rotated as it is to return to the front position, where the fixing is released. The control of fixing / releasing is performed, for example, by gazing at a control button displayed in the viewer screen (FIGS. 12A and 12B). By performing such a procedure, a specific object can be operated in an easy posture of a front position.

Further, as a modified example of the control, as shown in FIG. 13, near the front, information in the information space according to the rotation of the head mounted display 100 is displayed. When the rotation angle of the head-mounted display 100 exceeds a certain angle, the control may be such that the information in the information space is scrolled and displayed even if the head-mounted display 100 is fixed.

Here, k _3S and k _3P shown in Expressions 13 and 14 may be 1 or other values. Here performs scroll absolute value of the rotation angle lambda ₀ or more in the horizontal direction to perform the scroll in the absolute value of the rotation angle mu ₀ or more for a vertical direction. If the conditions are met, scroll both horizontally and vertically.

The scroll speed may be changed according to the rotation angle of the head mounted display 100. As shown in Expressions 15 and 16, for example, the scroll speed is increased as the angle increases (FIG. 14). For example, the display controller 72 converts the rotation angle into an angle larger than the rotation angle detected by the detector 71, and sends the angle to the information processing device 200.

The object extraction controller 82 extracts a display image in the information space corresponding to an angle larger than the rotation angle of the head mounted display 100, and sends the display image to the head mounted display 100. The display controller 72 receives the display image and displays the display image at a higher speed than the rotation angle. As described above, the display controller 72 additionally displays an object having a position exceeding the range of the degree of rotation as a display reference position.

によって Depending on the control method described above, the reference position in the information space (position at an angle of 0 degree) and the reference position in the virtual display space (position at an angle of 0 degree) may be shifted. In such a case, an operation of returning to the original reference position may be performed.

仮想 Alternatively, the virtual display space may be divided into regions, and the above control may be changed for each region (FIG. 15). The information space to be displayed is different for each area, and the information space to be controlled is changed depending on the direction of the head mounted display 100 in the virtual display space. An area for performing a scroll operation may be provided near the area boundary of the virtual display space. In addition, the boundary of the area or the scroll operation area may be displayed in the virtual display space, and further, a cursor may be displayed to indicate which direction the head-mounted display 100 faces in the virtual display space. .

Furthermore, the range (Δ ′ _S , Δ ′ _P ) of the information space displayed in the virtual display space may be different from the viewing angle (Δ _S , Δ _P ) of the virtual display space. For example, in a region where the rotation angle is large according to the rotation angle of the head-mounted display 100, a region that is equal to or larger than the viewing angle in the virtual display space is displayed. In FIG. 16, when the magnitude of the rotation angle is λ ′ ₀ or less, δ ′ _S = δ _S (that is, the display target range of the information space is equal to the viewing angle of the virtual display space), and the magnitude of the rotation angle is The example in which the display range of the information space is increased in an area larger than λ ′ ₀ has been described.

Based on the example of FIG. 16, the equations of δ ′ _S and δ ′ _P are shown in the following equations 17 and 18. If the display range of the information space is larger than the viewing angle of the virtual display space, the information in the information space is compressed and displayed in the virtual display space.

For example, when a display image having a rotation angle or more is obtained, the display controller 72 displays the display image in a compressed state. As described above, the display controller 72 compresses and displays the object to be displayed.

As described above, in the head-mounted display 100, the detector 71 detects the degree of rotation of the head-mounted display 100, and the display controller 72 displays the object at the display reference position exceeding the rotation angle. As described above, since the head mounted display 100 displays the object at the display reference position exceeding the rotation angle, it is possible to efficiently search for the object.

In the head-mounted display 100, the detector 71 detects any one of a rotation angle, an angular velocity, and an angular acceleration as the degree of rotation of the head-mounted display 100. That is, the head-mounted display 100 displays the object at the display reference position exceeding the rotation angle according to the degree of rotation, so that the object can be searched efficiently.

In the head mounted display 100, when the degree of rotation is equal to or more than the predetermined value, the display controller 72 displays the object at the display reference position exceeding the rotation angle. In this case, when it is considered that the wearer does not carefully refer to the object, such as when the rotation speed is high, the object at the display reference position exceeding the rotation angle is displayed, so when the wearer wants to refer to more objects Can refer to more objects.

In addition, in the head mounted display 100, the display controller 72 compresses and displays the display target object, so that more objects can be displayed and output.

(Example 2)
Embodiments 2 and 3 are embodiments using a cylindrical information space, and are hereinafter simply referred to as an information space. FIG. 17 is an explanatory diagram of a virtual display space displayed on the head mounted display 100. The virtual display space has a mapping relationship with the information space, but has the following features.
(1) The plane display has a shape between B and C, which is a range of left and right rotation angles ± δ around the Az axis with respect to a reference direction initially set by the wearer of the head mounted display 100 facing the front. This is the same display screen as a normal PC display in executing information processing, and the visibility of the curved screen is improved.
(2) In a space in which the wearer of the head-mounted display 100 faces the front and rotates left and right around the Az axis with respect to the reference direction initially set to ± δ or more, the image of the angle α of the virtual display space is the angle of the information space ( α * n) image (lateral direction) is mapped. Thus, information in a range of n times the viewing angle can be identified, and high-speed information retrieval can be performed.
Specifically, the following equation is used.

Here, V is a virtual display space, F is a mapping function from the information space, and n is a positive real number. Details of the mapping function are omitted because they are well-known techniques of coordinate transformation.

In this embodiment, the lateral direction of the visual field V in the reference direction facing the front coincides with the line segment BC. When the head is rotated right by δ ° and the front direction of the display 103 is directed to C, the left half of the visual field V becomes a plane image of the application A and the right half becomes a curved image of the application B curved into a cylindrical shape. FIG. 18 schematically illustrates this situation. Here, it is difficult to understand in FIG. 17, but as shown in FIG. 18, the mapping is performed so that the plane portion faces orthogonally to the front direction of the display 103.

FIG. 19 is an explanatory diagram showing an example in which a high-speed search can be performed in the virtual display space displayed on the head mounted display 100. In the case (a), the application 4 indicating the front visual field V is displayed as a planar image. In the cases (b) to (d), the head is rotated and the left end of the visual field V is at an angle θ.

アプリケーション One application display screen corresponds to the visual field V of the information space. The case (b) is a case where n = 1, and the display image of the application 5 is displayed as a curved surface. The case (c) is a case where n = 2, and the application 5 and the application 6 in the information space are double-compressed in the rotation angle direction and displayed on the visual field V in the virtual display space. The case (d) is a case where n = 3, and similarly, in the visual field V of the virtual display space, the application 5, the application 6, and the application 7 in the information space are three times compressed in the rotation angle direction and are displayed on a curved surface. . By compressing and displaying the fields of view of a plurality of information spaces in the field of view V of the virtual display space when the head is rotated in this manner, information can be searched at high speed.

For example, the display controller 72 converts the rotation angle into an angle larger than the rotation angle detected by the detector 71, and sends the angle to the information processing device 200.

The object extraction controller 82 extracts a display image in the information space corresponding to an angle larger than the rotation angle of the head mounted display 100, and sends the display image to the head mounted display 100. The display controller 72 displays a predetermined area in the display image in a curved state. As described above, the display controller 72 displays at least a part of the display target object in a curved manner. Accordingly, the head mounted display 100 can display the central portion of the display 103 (the portion considered to be watched by the wearer) in a planar manner, and can display the other portions in a curved manner.

In this figure, n is an integer for ease of explanation, but is not limited to this and may be a real number (preferably 1 or more from the viewpoint of high-speed search). Although the value of n is also a fixed value, it is also possible to set a plurality of values corresponding to a specific rotation angle. For example, the range from the angle δ to δ + 15 ° is set to n = 2, the range from the angle δ + 15 ° to δ + 30 ° is set to n = 3, and the range from the angle δ + 30 ° or more is set to n = 4. is there.

If the rotation angle of the head is increased, the compression ratio from the information space to the virtual display space can be further increased.

(Processing procedure of the above embodiment)
Hereinafter, the operation of the head mounted display 100 of the above-described embodiment will be described. In the following description, the case where the wearer's head is rotated clockwise is described. However, it is apparent that the same processing can be executed when the wearer's head is rotated counterclockwise.

FIG. 20 is a flowchart showing an operation of displaying information in the information space on the head mounted display 100.

First, with the head of the wearer wearing the head-mounted display 100 facing the front, the input operation unit 107 initially registers the reference direction shown in the front direction shown in FIG.

(4) Information display is started by the head mounted display 100. In step S701, the detector 71 acquires (detects) information of the attitude sensor unit 110. Specifically, information such as the attitude change of the display 103, the direction of the change, the amount of the change, and the like is obtained by using the acceleration sensor 111, the gyro sensor 112, and the geomagnetic sensor 113 arranged at appropriate positions on the display 103 of the head mounted display 100. To get.

In step S702, the display controller 72 obtains the zoom magnification stored in the information processing device 200 and performs sensitivity adjustment. As the zoom magnification increases, the sensitivity of the head angle detection (lateral direction) decreases. This is because, when zooming, the angle of view becomes smaller, and by lowering the head angle detection sensitivity, the vibration of the displayed image due to the shaking of the head can be suppressed.

In step S703, the display controller 72 acquires the display direction information (corresponding to the head direction information) after the sensitivity adjustment, and transmits the information to the information processing apparatus 200 via the communication unit 120. Note that the display unit direction information includes a rotation angle (head rotation) and an elevation angle (head up / down) with respect to the reference direction that is initially registered. That is, the display controller 72 transmits the display unit direction information to the information processing device 200 as the information indicating the degree of rotation.

Next, in step S704, the object extraction controller 82 of the information processing device 200 acquires the display direction information via the communication unit 220 and the information space information stored in the storage 204. Here, the home position of the information space, that is, the front visual field V (the display image of the application 4 in the example of FIG. 6B) is the screen saved at the end of the previous time or the desktop screen of a normal PC at the time of new startup. Is displayed.

(4) Each time the application is started, a new tile is added to the information space. It is assumed that a processing procedure is set in advance at which position of a tile in the information space to be sequentially added.

In step S705, the object extraction controller 82 sets the camera direction of the viewer with respect to the information space based on the display unit direction information. The image of the viewer is equivalent to the field of view V.

Next, in step S706, the object extraction controller 82 extracts a display image in the range of the angle of view of the viewer from the information space. Details will be described later. The extracted image is transmitted to the head mounted display 100 via the communication unit 220.

In step S707, the display controller 72 receives the extracted image through the communication unit 120 and displays the image on the display 103 of the head mounted display 100. In step S709, it is determined whether the process has been completed.

In step S709, if the display controller 72 determines that the processing has not been completed (step S709: No), the process returns to step S701, and the detector 71 acquires the posture sensor information again. In step S709, when the display controller 72 determines that the process is completed (step S709: Yes), the process is completed.

In step S708, the information processing device 200 determines whether there is a scroll operation input. The scroll operation input is not particularly limited to operation input means, such as operating the keyboard 300, inputting by voice, or instructing by a controller held in the hand, and may be any operation that can be recognized as a scroll operation. For example, the accepting unit 73 may receive the information indicating that the object has been accepted and the target object, and the information processing apparatus 200 may scroll in response thereto.

If it is determined in step S708 that there is no scroll operation input (step S708: No), the process ends, and if it is determined in step S708 that there is a scroll operation input, the information space is scrolled (step S710). I do.

Here, scrolling of the information space means moving the image of the viewer set in step 705 to the home position, and moving (rotating) the information space accordingly.

Next, the process of extracting the display image in the range of the angle of view of the viewer from the information space in step S706 will be described in detail with reference to FIG. FIG. 21 is a flowchart illustrating a process of extracting a display image in a range between both corners of the viewer from the information space.

In step S801, it is determined whether the angle δ is included in the angle of view of the viewer. The angle δ is included when the rotation angle α from the center line of r ′ in the virtual display space shown in FIG.

In step S801, when the object extraction controller 82 determines that the angle δ is included in the angle of view of the viewer (step S801: Yes), the object extraction controller 82 performs mapping conversion from the information space to the virtual display space in step S803. Specifically, in the area where the rotation angle in the viewer is smaller than δ, the image data of the information belt is mapped to the plane image data in the virtual display space. In the area where the rotation angle in the viewer is larger than δ, the image data in the information space is converted. Is compressed n times in the circumferential direction and is mapped to the virtual display space. Therefore, the image is a combined image of the plane image and the curved surface image compressed in the circumferential direction.

In step S801, when the object extraction controller 82 determines that the angle δ is not included in the angle of view of the viewer (step S801: No), the object extraction controller 82 determines in step S802 whether or not the entire angle of view of the viewer is larger than δ. .

In step S802, if the object extraction controller 82 determines that all are greater than δ (step S802: Yes), the object extraction controller 82 performs mapping transformation from the information space to the virtual display space in step S804. Specifically, the image data in the information space is compressed n times in the circumferential direction and is mapped to the virtual display space. Therefore, only a curved surface image compressed in the circumferential direction is obtained.

If the object extraction controller 82 determines in step S802 that the value is not larger than δ (step S802: No), the object extraction controller 82 performs mapping conversion from the information space to the virtual display space in step S805. Specifically, the object extraction controller 82 performs mapping conversion of the image data in the information space to planar image data in the virtual display space. Therefore, only a planar image is obtained.

Next, in step S806, the object extraction controller 82 extracts image display data within the range of the viewing angle of the viewer from the virtual display space, and ends the processing.

(Example 3)
In the present embodiment, a display method in a curved surface area in the virtual display space is different from that in the second embodiment. In the first embodiment, the image data in the information space is displayed in the curved surface area by compressing it in the circumferential direction by n times. In the present embodiment, the image data is not compressed and a scroll (rotated) image is displayed. It is.

FIG. 22 is an operation flowchart of the head mounted display 100 according to the third embodiment. The same parts as those in FIG. 20 are denoted by the same reference numerals, and detailed description thereof will be omitted.

において FIG. 22 differs from the processing of FIG. 20 in that steps S708 and S710 are deleted. Further, similar to the first embodiment, in the following description, the case where the head is rotated clockwise is described, but the same applies to the case where the head is clockwise.

Next, the process of extracting the display image in the range of the angle of view of the viewer from the information space in step S706 will be described in detail with reference to FIG.

In step S101, the object extraction controller 82 determines whether the angle of view δ is included in the angle of view of the viewer. The angle δ is included when the rotation angle α from the center line of r ′ in the virtual display space shown in FIG.

In step S101, if the object extraction controller 82 determines that the angle δ is included in the angle of view of the viewer (step S101: Yes), the object extraction controller 82 switches the information space from the information space to the virtual display space in step S103. Perform a mapping transformation. Specifically, in the area where the rotation angle in the viewer is smaller than δ, the image data of the information belt is mapped to the plane image data in the virtual display space. In the area where the rotation angle in the viewer is larger than δ, the image data in the information space is converted. Is scrolled to the left at the angular velocity γ to perform mapping conversion to the virtual display space.

Therefore, an image is obtained by combining the plane image and the left scroll image. The plane image is not scrolled. It is assumed that the wearer of the head mounted display 100 appropriately sets and changes the angular velocity γ using the input operation device 107 and the like.

When the object extraction controller 82 determines in step S101 that the angle δ is not included in the angle of view of the viewer (step S101: No), the object extraction controller 82 determines in step S102 that the angle δ is all within the angle of view of the viewer. Determine if it is greater than.

If the object extraction controller 82 determines in step S102 that all are greater than δ (step S102: Yes), mapping transformation from the information space to the virtual display space is executed in step S104. Specifically, the image data in the information space is scrolled to the left at the angular velocity γ, and is mapped to the virtual display space. Therefore, only the left scroll curved surface image is displayed.

{Circle over (4)} In step S102, when the object extraction controller 82 determines that none of them are larger than δ (step S102: No), the mapping conversion from the information space to the virtual display space is executed in step S105. Specifically, the image data in the information space is mapped and converted into the plane image data in the virtual display space. Therefore, only a planar image is obtained.

(5) Next, in step S106, image display data within the range of the viewing angle of the viewer is extracted from the virtual display space.

In the present embodiment, the value of the angular velocity γ is a fixed value, but a plurality of values can be set corresponding to a specific rotation angle. For example, the range from the angle δ to δ + 15 ° is set as angular velocity γ = 0.1, the range from the angle δ + 15 ° to δ + 30 ° is set as angular velocity γ = 0.3, and the range above the angle δ + 30 ° is set as angular velocity γ = 0.8. And so on. In this way, when the rotation angle of the head is increased, the rotation speed of the information space is further increased, and the amount of information that can be observed in a short time can be increased.

In step S107, the object extraction controller 82 determines whether there is a scroll image selection input. Here, scroll image selection input means that a viewer image at a certain point in time is selected from images scrolling in the viewer. If it is determined in step S107 that there is a selection input (step S107: Yes), the image selected in step S108 is moved to the home position, and the information space is moved (rotated) accordingly. If it is determined in step S107 that there is no selection input (step S107: No), the process ends.

Therefore, when the head is rotated rightward from the direction of the home position, a range equal to or larger than the set angle δ appears to scroll leftward. Here, when the head is returned to the home position, the image of the original home position can be observed. This is because the image at the home position is not changed unless scroll selection input is performed.

異なる When the head is rotated counterclockwise, scrolling to the right in the range of the angle -δ is different from the clockwise rotation.

Next, with reference to FIG. 24, a processing procedure for performing a display process in the head-mounted display 100 or a system in which the head-mounted display 100 and the information processing device 200 are integrated will be described. The detector 71 of the head-mounted display 100 acquires the posture sensor information, and the display controller 72 updates the direction information of the HMD (Step S201). The display controller 72 updates the rotation angular velocity information of the HMD based on the posture sensor information (Step S202).

The display controller 72 calculates the direction range of the information space to be displayed from the updated direction information of the HMD and the rotational angular velocity information, transmits the direction range to the information processing device 200, and transmits the direction range from the information processing device 200 to the information processing device 200. A display image based on the range is received (step S203).

(4) The display controller 72 updates the display of the virtual display space with the updated information of the information space to be displayed (the display image) (step S204).

The display controller 72 determines whether there is an instruction to end the display processing (step S205), and when there is no instruction to end the display processing (step S205: No), proceeds to step S201. If there is an instruction to end the display processing (step S205: Yes), the display controller 72 ends the processing.

Although the example of the embodiment of the present invention has been described above, it is needless to say that the configuration for realizing the technology of the present invention is not limited to the above-described example, and various modifications are possible. For example, when using the operation means, an external operation menu is confirmed through a see-through type head mounted display, but the present invention is not limited to this. For example, the operation menu may be superimposed on the display unit in a non-see-through type head mounted display.

Furthermore, in a range exceeding a specific angle of the visual field V, information corresponding to n times the visual field angle in the information space is displayed or rotation information is displayed, but the present invention is not limited to this. For example, the field of view V is divided. For example, information in the information space is displayed as it is in the upper divided area, and information corresponding to n times the viewing angle in the information space in a range exceeding a specific angle of the field of view V in the lower divided area. May be displayed, or rotation information may be displayed. Further, numerical values and the like appearing in the text and figures are merely examples, and the use of different values does not impair the effects of the present invention.

Further, in each of the above-described embodiments, the case where the head mounted display 100 and the information processing device 200 are dispersed has been described, but an integrated device may be used.

The functions and the like of the present invention described above may be partially or wholly realized by hardware, for example, by designing an integrated circuit. Alternatively, the software may be realized by a microprocessor unit or the like interpreting and executing a program for realizing each function or the like. Hardware and software may be used together.

制御 Also, the control lines and information lines shown in the figure indicate those which are considered necessary for explanation, and do not necessarily indicate all the control lines and information lines on the product. In fact, it can be considered that almost all components are connected to each other.

Reference numeral 71: detector, 72: display controller, 73: receiver, 81: memory, 82: object extraction controller, 100: head mounted display, 101: controller, 102: memory, 103: display, 104: GPS receiver, 105 ... voice input unit, 106 ... voice output unit, 107 ... input operation unit, 110 ... attitude sensor unit, 111 ... acceleration sensor, 112 ... gyro sensor, 113 ... geomagnetic sensor, 120 ... communication unit, 121 ... LAN communication unit, 122 ... Telephone network communication device, 123 ... Near field communication device, 140 ... Camera, 200 ... Information processing device, 201 ... Controller, 202 ... Memory, 203 ... Display, 204 ... Storage, 205 ... Input operation device, 220 ... Communication unit 221 LAN communication device 222 telephone network communication device 223 near Away wireless communication device, 300 ... keyboard.

Claims

A head-mounted display that performs display control of an object whose display reference position is determined in advance,
A detector for detecting the degree of rotation of the head mounted display,
Based on the rotation degree detected by the detector, a display controller that displays an object at the display reference position that exceeds a rotation angle,
Head-mounted display with
The head mounted display according to claim 1, wherein
The detector detects any one of a rotation angle, an angular velocity, and an angular acceleration as a rotation degree of the head-mounted display,
Head mounted display.
The head mounted display according to claim 1 or 2,
The display controller, when the degree of rotation is equal to or more than a predetermined value, displays an object at the display reference position that exceeds a rotation angle,
Head mounted display.
The head mounted display according to any one of claims 1 to 3, wherein
The display controller compresses and displays at least a part of the display target object,
Head mounted display.
The head mounted display according to any one of claims 1 to 4,
The display controller curvedly displays at least a part of the display target object,
Head mounted display.
The head mounted display according to any one of claims 1 to 5, wherein
The display controller further includes a reception unit that receives designation of an object to be displayed,
The display controller fixedly displays the object whose specification has been received by the reception device,
Head mounted display.
An object for which a display reference position is determined in advance, a method of displaying an object whose head-mounted display controls display,
A detecting step of detecting the degree of rotation of the head mounted display,
A display control step of displaying an object at the display reference position exceeding a rotation angle based on the rotation degree detected in the detection step;
How to display objects that contain.