JP6535819B2 - Control system for navigation in virtual reality environment - Google Patents

Description

  The present application relates generally to movement and scaling in augmented reality and / or virtual reality environments.

  Augmented reality (AR) and / or virtual reality (VR) systems may create a three dimensional (3D) immersive virtual environment. Various electronic devices, such as, for example, helmets or other head-mounted devices, including displays, glasses, or goggles, which the user looks into when viewing the display device, gloves with attached sensors, external hand-held devices including sensors, and others Through interaction with such electronic devices, the user can experience this 3D immersive virtual environment. When immersing in a 3D virtual environment, the user travels through the virtual environment through the operation of physical devices and / or electronic devices to interact with and personalize the interaction with the virtual environment, the virtual environment You may move to another area of

FIG. 16 is a diagram of an example implementation of a virtual reality system that includes a head mounted display device and a controller according to the implementations described herein. FIG. 2 is a perspective view of an exemplary head mounted display. FIG. 2 is a perspective view of an exemplary head mounted display. FIG. 7 illustrates a controller according to the implementations described herein. FIG. 6 is a block diagram of head mounted electronics and a controller according to the implementations described herein. A third person view that illustrates navigation and scaling (scale change) in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. A third person view that illustrates navigation and scaling in an augmented and / or virtual reality environment according to the implementations described herein. FIG. 7 illustrates movement with a virtual photosphere in an augmented and / or virtual reality environment according to implementations described herein. FIG. 7 illustrates movement with a virtual photosphere in an augmented and / or virtual reality environment according to implementations described herein. 10 is a flowchart of a method of navigating and scaling in an augmented and / or virtual reality environment according to implementations described herein. FIG. 10 is an illustration of an example computing device and mobile computing device that can be used to implement the techniques described herein.

  In one aspect, a method includes: generating a virtual environment; detecting a first input at a user interface of a controller in communication with a head mounted display device; and responding to the first input in the virtual environment Setting an anchor point to the selected virtual feature, detecting a second input, defining an area of the feature surrounding the anchor point in response to the second input, and viewing the user of the virtual environment Adjusting at least one of the position and the scale of the virtual feature in the virtual environment while maintaining a part of the virtual feature within the defined area.

  In another aspect, a system comprises a computing device configured to create an immersive virtual environment, the computing device having a memory for storing executable instructions and a processor configured to execute the instructions. You may The processor is operable to: generate a virtual environment on the computing device; detecting a first input at a user interface of a controller in communication with the head mounted display device; and selecting in the virtual environment in response to the first input. Setting an anchor point to the virtual feature, detecting a second input, defining an area of the feature surrounding the anchor point in response to the second input, and within the user view of the virtual environment Adjusting at least one of the position and the scale of the virtual feature in the virtual environment while maintaining a part of the virtual feature in the defined area.

The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.
A user immersed in a 3D virtual environment, for example, wearing a head mounted display (HMD) device may explore the 3D virtual environment through various different types of inputs and may interact with the 3D virtual environment. These inputs may be, for example, head movement and / or directional gaze of the head and / or eyes, etc. through manipulation of the electronic device separate from the HMD, manipulation of the HMD itself and / or hand / arm gestures, etc. May include, for example, physical interaction. The user implements one or more of these different types of interactions to perform specific actions in the virtual environment, such as moving in the virtual environment or moving the virtual environment with respect to the user, from the first area of the virtual environment Moving, transitioning or teleporting to a second area of the virtual environment, adjusting the perspective in which the virtual environment is experienced, etc. may be performed.

  Systems and methods according to implementations described herein may operate on features in a virtual environment, may move or navigate through the virtual environment, and users view virtual environments from different perspectives and scales. You may be able to The systems and methods according to the implementations described herein also provide for the disconnection of the dynamic visual movement experienced in virtual reality and the lack of actual physical movement corresponding to the dynamic visual movement. A substantially seamless virtual movement experience in virtual reality may be provided to the user while avoiding the often associated motion sickness and loss of orientation.

  In the exemplary implementation shown in FIG. 1, a user wearing the HMD 100 has a handheld electronic device 102. Handheld electronic device 102 may be, for example, a controller that may be paired with and communicate with HMD 100, a gyro mouse, a smartphone, a joystick, or another portable controller for interaction in an immersive virtual environment generated by HMD 100. In the following, the handheld electronic device 102 will be referred to as the controller 102 merely for ease of discussion and explanation. The controller 102 may be operably coupled or paired with the HMD 100 via, for example, a wired connection or a wireless connection such as, for example, a WiFi or Bluetooth® connection. This pairing or operable coupling of the controller 102 and the HMD 100 may provide for communication between the controller 102 and the HMD 100 and exchange of data between the controller 102 and the HMD 100. Thus, the controller 102 may be able to function as a controller communicating with the HMD 100 in order to interact in the immersive virtual environment generated by the HMD 100. That is, controller 102 may be operated in a number of different ways. The operations of controller 102 may be interpreted in the immersive virtual environment generated by HMD 100 as corresponding selections or movements or other types of interactions. This includes, for example, interaction with, manipulation or coordination with virtual objects, changes in scale or perspective with respect to the virtual environment, and movement of the user from the current location in the virtual environment to the selected destination or feature in the virtual environment (eg navigation , Teleportation, transport) and other such interactions. In some implementations, as discussed above, the user has physical gestures such as virtual features in a virtual environment and hand gestures detected by a system equipped to detect and track the user's hands. You may interact without relying on the use of a separate controller.

  2A and 2B are perspective views of an exemplary HMD, such as, for example, the HMD 100 worn by the user in FIG. FIG. 2C is a diagram illustrating an exemplary controller, such as, for example, the controller 102 shown in FIG.

  The controller 102 may include a housing 103 in which the internal components of the device 102 are housed, and a user interface 104 external to the housing 103 and accessible by a user. The user interface 104 includes, for example, a touch-sensitive surface 106 configured to receive user touch input, and an operating device 105 including buttons, knobs, joysticks, toggles, slides, and other such operating devices. Several different types of operating devices may be included. In some implementations, controller 102 is also configured to selectively emit light source 108, eg, a beam or beam, through a port in housing 103, for example, in response to user input received at user interface 104. May be included.

  The HMD 100 may include a housing 110 connected to the frame 120 and an audio output device 130 also connected to the frame 120, including, for example, speakers mounted on headphones. In FIG. 2B, the front portion 110a of the housing 110 is rotated away from the base portion 110b of the housing 110 so that some of the components contained in the housing 110 can be seen. The display 140 may be mounted on the interior facing side of the front portion 110 a of the housing 110. The lens 150 may be mounted in the housing 110 between the user's eye and the display 140 when the front portion 110 a is in a closed position relative to the base portion 110 b of the housing 110. In some implementations, the HMD 100 may include a sensing system 160 that includes various sensors such as, for example, audio sensors, image / light sensors, position sensors (eg, inertial measurement devices including gyroscopes and accelerometers). HMD 100 also includes a control system 170 that includes processor 190 and various control system devices to perform the operations of HMD 100.

  In some implementations, the HMD 100 may include a camera 180 that captures still and moving images. The images acquired by the camera 180 may be used to assist in tracking the physical location of the user and / or controller 102 in the real world, and / or may be displayed to the user on the display 140 in pass-through mode, the HMD 100 Allow the user to temporarily leave the virtual environment and return to the physical environment without changing the configuration of the HMD 100 to move the housing 110 out of the user's line of sight, such as.

  In some implementations, the HMD 100 may include a gaze tracking device 165 that detects and tracks gaze of the user's eyes. The gaze tracking device 165 acquires an image of the user's eye, for example an image of a particular part of the user's eye, such as the pupil, and detects and tracks the user's gaze direction and movement, for example a single image sensor 165A or A plurality of image sensors 165A may be included. In some implementations, the HMD 100 may be configured such that the detected gaze is treated as user input and interpreted into corresponding interactions in an immersive virtual experience.

  A block diagram of a system for providing navigation manipulation and control in an augmented and / or virtual reality environment is shown in FIG. The system may include a first electronic device 300 in communication with the second electronic device 302. The first electronic device 300 may be, for example, an HMD, as described above with respect to FIGS. 1, 2A and 2B, to create an immersive virtual environment. The second electronic device 302 may be, for example, a controller, as described above with respect to FIGS. 1 and 2C, to perform user interaction with the immersive virtual environment generated by the first electronic device 300. It communicates with the device 300.

  The first electronic device 300 may include a sensing system 360 and a control system 370 that may be similar to the sensing system 160 and the control system 170 shown in FIGS. 2A and 2B, respectively. The sensing system 360 may be, for example, an optical sensor, an audio sensor, an image sensor, a distance / proximity sensor, a position sensor (eg, an inertial measurement device including a gyroscope and an accelerometer), and / or other sensors, and / or FIG. One or more different types of sensors may be included, including different combinations of sensors including, for example, an image sensor installed to detect and track the user's eye gaze, such as the gaze tracking device 165 shown. . Control system 370 may include, for example, a power / pose control device, an audio and video control device, an optical control device, a transition control device, and / or other such devices, and / or different combinations of devices. Sensing system 360 and / or control system 370 may include more or less devices, depending on the particular implementation. Elements included in sensing system 360 and / or control system 370 may have different physical arrangements (eg, different physical locations) within an HMD other than HMD 100 shown, for example, in FIGS. 2A and 2B. The first electronic device 300 also communicates with the processor 390 in communication with the sensing system 360 and the control system 370, the memory 380, and the first electronic device 300 and another external device, such as the second electronic device 302. A communication module 350 may be included to provide.

  The second electronic device 302 may include a communication module 306 that provides communication between the second electronic device 302 and another external device, such as the first electronic device 300. In addition to providing exchange of data between the first electronic device 300 and the second electronic device 302, the communication module 306 may also be configured to emit a light beam or beam as described above. The second electronic device 302 may for example be a sensing system 304 including an image sensor and an audio sensor such as included in a camera and a microphone, an inertial measurement device, a touch sensor as included in a controller or a touch sensitive surface of a smartphone, and the like Such sensors, and / or different combinations of sensors may be included. Processor 309 may be in communication with sensing system 304 and control unit 305 of second electronic device 302. Control unit 305 has access to memory 308 and controls the overall operation of second electronic device 302.

  As mentioned above, a controller, such as the controller 102 described above, may be manipulated by the user, often in combination with the functionality of the HMD 100 described above, for interaction and navigation in a virtual environment. Exemplary implementations of this are shown in FIGS. 4A-4E.

  A user wearing the HMD 100 creating a virtual environment experienced by the user may operate the controller 102 to navigate and manipulate virtual objects and virtual features in the virtual environment 400. As shown in FIG. 4A, a fixed, configured controller reference point 410 may remain fixed with respect to the controller 102 when the controller 102 moves, but may be associated with the controller 102. In the following, the set controller reference point 410 is illustrated as a set of virtual crosshairs in the drawings, merely for the sake of simplicity and discussion. The set controller reference point 410 exists at the intersection of the virtual crosshairs. The configured controller reference point 410 / virtual cross hair 410 may move with the controller 102 and remain in a fixed position with respect to the controller 102 as the user moves the controller 102 relative to virtual objects and features in the virtual environment 400. In the example shown in FIG. 4A, virtual crosshair 410 includes a virtual representation, or icon, that remains in a fixed position relative to controller 102 and essentially moves with controller 102. The virtual crosshairs 410 are represented as such in the following merely for ease of discussion and explanation. However, in some embodiments, the virtual crosshair 410 may represent a known position in space relative to the controller 102 (detected by the controller 102 and / or the HMD 100), but the visual representation or icon to the user The view and access of the user to virtual objects and features in the virtual environment 400 are not obscured by the visual representation of the virtual crosshair 410.

  FIG. 4A illustrates the initiation of a virtual interaction with a user virtual feature 450. Virtual feature 450 is shown as seen by the user at the start of the virtual interaction. As shown in FIG. 4A, at the start of the user's interaction with virtual feature 450 in virtual environment 400, eg, at time t = 0, the user may define anchor point 420 associated with virtual feature 450. In FIG. 4A, anchor point 420 may want the user to stay in his view, for example, as the user moves in virtual environment 400 and / or changes position and / or perspective relative to virtual objects and features. Portions of virtual feature 450 may be identified. The anchor point 420 may be defined by the virtual target line 415 at the beginning of the user's interaction with the virtual feature 450, ie at time t = 0. The virtual target line 415 extends from the set user reference point 100A and is defined by the direction of the virtual target line 415 which may be controlled by the user reference point 100A and / or the user moving the virtual cross hair 410. The selected part intersects with the virtual cross hair 410.

  The user reference point 100A is defined at a set place in the HMD 100, such as a position at or near the user's eye, a position corresponding to a place between the user's eyes, or another set place. Good. The set user reference point 100A remains set on the HMD 100 or with respect to the HMD 100 when the user and in particular the user's head move in the virtual environment 400 to change the position / orientation, ie, it remains constant. Good. Similarly, the location / position / orientation of the virtual crosshair 410 relative to the controller 102 may be on or on the controller 102 as the user and in particular the hand holding the user's arm / controller 102 changes location / position / orientation in the virtual environment. It may be set to 102, that is, it may remain at a fixed position. The location and position / direction of the HMD 100 and the location and position / direction of the controller 102 relative to the HMD 100 may be detected and tracked by the system in substantially real time, relative to the fixed location of the set user reference point 100A and the controller 102. Certain locations of virtual crosshairs 410 may also be detected and tracked in substantially real time.

  In the example shown in FIG. 4A, the virtual target line 415 extending from the set user reference point 100A to the selected anchor point 420 through the virtual cross hair 410 is represented by a dotted line. However, in some implementations, the virtual target line 415 may not necessarily be rendered as a visual representation or object seen by the user, so that the user's view and access to virtual objects and features in the virtual environment is a virtual target line Not obscured by the 420 visual representation. In some implementations, a visible representation, eg, a dot, may be rendered as anchor point 420 at a point on virtual feature 450 identified by virtual target line 415. An example of the user's view, ie, the perspective from the user's virtual position relative to the virtual feature 450 shown in FIG. 4A when the user sets the anchor point 420, is shown in FIG. 4B. The configuration shown in FIG. 4A represents the beginning of the user's interaction with the virtual feature 450 by operating the device at the user interface 104 of the controller 102, eg pressing a button or trigger, ie, time t = 0 You may

  The depression of a button or trigger (or activation of another operating device in the user interface 104 of the controller 102) may be used to identify and set the anchor point 420, as shown in FIG. 4A. After setting the anchor point 420, the user's movements, eg head and hand / arm movements, and the corresponding movement of the virtual target line 415 extending over the virtual cross hair 410 to the anchor point 420 (above the user's head The movement of the set user reference point 100A and the movement of the controller 102 held by the user's hand) may move and scale the virtual feature 450 in the virtual environment 400. For example, when anchor point 420 is set as described above, downward movement (from the position shown in FIG. 4A with the button or trigger held down) of virtual target line 415 as shown in FIG. A downward movement of point 420 may occur, a change in the scale of virtual environment 400 (while the user's foot remains on virtual ground, and virtual feature 450 remains on virtual ground), and, in particular, the virtual environment This results in a change in the scale of the virtual feature 450 anchored by the anchor point 420 at 400 and the user's view or perspective on the virtual feature 450 as shown in FIG. 4D.

  Similarly, once anchor point 420 is set as described above, upward movement of virtual target line 415 as shown in FIG. 4E (eg, from the position shown in FIG. 4A with the button or trigger held down) , May cause the upward movement of anchor point 420, the change in scale of virtual environment 400, and in particular, the user's scale or perspective relative to virtual feature 450 anchored by anchor point 420 in virtual environment 400, and shown in FIG. 4F. Result in a change in the user's view or perspective on the feature.

  Adjusting the scale in the virtual environment may be a change (increase or decrease) in the size of the user for the virtual feature in the virtual environment or a corresponding change in the user's perspective for the virtual feature in the virtual environment , May be considered as an increase or decrease in the size of virtual features for users in the virtual environment). For example, as the user sees the virtual environment from the user's perspective as if his / her size for the virtual features in the virtual environment has increased (and / or the size / scale of the virtual features may appear to decrease) You may choose to scale up as you experience. Similarly, the user can view the virtual environment as if from the user's perspective, as if his own size decreased for virtual features in the virtual environment (and / or the virtual feature size / scale would increase) ) You may choose to scale down as you experience. This kind of scaling in the virtual environment may be considered as the size / scaling of the user, in particular the virtual adjustment in the user's perspective for virtual features in the virtual environment, or the virtual adjustment in the size / scaling of virtual features for the user in the virtual environment. In the following, scaling is considered to include virtual adjustment of the user's size / scale to virtual features and / or virtual adjustment of the virtual feature's size / scale in the virtual environment, merely to simplify the discussion. The position shown in FIG. 4E from the position shown in FIG. 4A to the position shown in FIG. 4C (shown in phantom as a ghost image in FIG. 4C) and / or from the position shown in FIG. Motions as illustrated by the dotted line) and the corresponding movement of the virtual features 450 in the virtual environment 400 and / or scaling of the user relative to the virtual environment 400 (or of the virtual environment 400 to the user). Describe the scaling). However, movement in the other direction (while the button or trigger is held down) also causes anchor point 420 to remain anchored to the identified portion of selected virtual feature 450 through the corresponding movement. Good. For example, movement to the right may cause the virtual feature 450 to move or orbit around the user. Similarly, movement of the user's arm closer to the user may produce the effect of zooming closer to the virtual feature 450.

  In response to an operation, the system may use a set user reference point, the position of virtual crosshair 410 relative to controller 102, and certain fixed parameters associated with set virtual reference point 400A. The virtual reference point 400A remains substantially stationary and does not change through movement of various motions and / or features and / or scaling as described above. Hereinafter, the set virtual reference point 400A is a reference plane or a virtual floor surface corresponding to a virtual ground in the virtual environment 400. That is, as illustrated in the example shown in FIG. 5, in response to an exemplary downward movement (from the arm / hand / controller position shown by the dotted line to the arm / hand / controller position shown by the solid line) The system uses certain fixed parameters to allow the user to rescale virtual feature 450 such that it is all within the scope of virtual target line 415, rescaled feature 450A, rescaled feature 450B, or rescaled feature It may be determined whether it is intended to be 450C. This will be described in more detail with reference to FIGS. 6A-6F.

In FIG. 6A, at the start of an interaction with virtual feature 450 in virtual environment 400 (e.g., at time t = 0), the user faces virtual feature 450 on virtual reference plane 400A, which is considered below as virtual ground 400A. May be located. In the example shown in FIG. 6A, the virtual target line 415 extends from the reference point 100A set by the user to the virtual feature 450 through the virtual cross hair 410 because the user stretches his arm and has the controller 102 at time t = 0. And set an anchor point 420. In some implementations, the size or dimensions (height and width) of virtual crosshair 410 may be constant or may be set, for example, for a particular application or by a user, system, or the like. Set the dimensions of the virtual crosshairs 410 may define a first disc d1 ₀ at time t = 0. The first disk d1 ₀ has a constant radius r1 defined by the set size of the virtual crosshairs 410. For example, at the beginning of the user's interaction with the virtual feature 450 by operating the device on the user interface 104 of the controller 102, such as pressing a button or trigger, ie at t = 0, to the user shown in FIG. 4A An example of what can be seen is illustrated in FIG. 6B.

The first disk d1 ₀ may define a cone 440. The cone 440 starts from the set user reference point 100 A and extends to contact the first disc d ₁₀ toward a plane surrounding the anchor point 420. at t = 0, the angle alpha ₀ may be defined as a semi-apex angle of the cone 440 defined by the user reference point 100A and the first disk d1 ₀ that has been set. The second disc d2 ₀ having a radius r2 passes through the distance corresponding to the anchor point 420, i.e., including the correspondence / to the anchor point 420, the virtual target line 415 substantially perpendicular virtual plane, a cone 440 It may be defined as a cross section.

When set as shown in FIG. 6A, virtual radius r 2 may remain fixed with respect to virtual environment 400, ie, constant with respect to virtual environment 400. That is, when set, has a certain virtual radius r2 to the virtual environment 400, the portion of the virtual feature 450 incorporated in the second disc d2 ₀ in the region, as shown in Figure 6B, the same can remain while the virtual feature 450 comprising a portion of the virtual feature 450 incorporated in the second disc d2 in _0, in response to the user moves the anchor point 420, is moved and / or scaled. This will be described in more detail with reference to FIGS. 6C-6F.

FIG. 6C illustrates the subsequent motion of virtual target line 415, which illustrates t> 0 and corresponds to the downward motion of anchor point 420 and the scaling down of virtual feature 450. An example of what can be seen by the user shown in FIG. 4C is illustrated in FIG. 6D after movement from the position shown in FIG. 6A to the position shown in FIG. 6C (described in more detail below). Figure 6E illustrates a later point in time, t> 0, corresponding to the approach movement of the anchor point 420, closer to the user reference point 100A that is set inside movement of the first disk d1 ₀ and the user and the virtual feature Explain the reduction of perceived virtual distance between 450 and 450. After movement from the position shown in FIG. 6A to the position shown in FIG. 6E (described in more detail below), an example of what can be seen by the user shown in FIG. 4E is illustrated in FIG. 6F.

  As described above with respect to FIG. 6A, after setting the anchor point 420, at each subsequent time t> 0 (eg, t1, t2,... TN), the angle α t is set to the set user reference point 100A and the first disc It may be calculated as the half apex angle of the cone 440 defined by d1t. At each subsequent time t> 0, the radius r1 of the first disc d1t remains constant / fixed to the user (when the size of the virtual cross hair 410 is constant). At each subsequent time t> 0, the virtual radius r2 of the second disk d2t remains constant / fixed relative to the virtual environment 400 when the anchor point 420 moves and remains on the virtual target line 415. The virtual reference plane 400A, i.e. the virtual ground 400A, also remains fixed or constant and remains consistent with the real world ground. However, the distance between the set user reference point 100A and the first disc d1t may change as the user's arm / hand with the controller 102 moves. This may also cause the semi-apical angle αt to also change in response to the user moving the anchor point 420 as described above at subsequent times t> 0 (eg, t1, t2, ... tN). The radius r1 of the first disk d1t remains constant to the user and the virtual radius r2 of the second disk d2t remains constant with respect to the virtual environment 400, the anchor point 420 remains on the virtual target line 415, and While the virtual reference plane 400A remains in line with the ground of the real world, as the anchor point 420 moves in this way, recalculating the half apex angle αt moves the virtual environment and / or as shown. Or allow to scale.

In the example shown in FIGS. 6A to 6E, the first disk d1 ₀ / dt ₀ displaying the area surrounding the set controller reference point 410 is a disk based on a virtual cross hair having a crossing point at the set controller reference point 410 Defined as By this exemplary geometry defined by a cone, the area surrounding the set controller reference point 410 has a circular disk shape and the area surrounding the anchor point 420, the area surrounding the anchor point 420 has a circular disk shape, As the user and / or virtual features are scaled in response to controller movement, the half apex angle α may be calculated and tracked / adjusted, and a reduced throughput may be generated. However, circular disks are shown merely for ease of discussion and explanation, and the area surrounding the set controller reference point 410 may have other closed curve shapes. Similarly, in the example shown in FIGS. 6A to 6E, a cone 440, a start point of the user reference point 100A which is set, elongation in contact with the first disk d1 _0, defined by the geometry of greater than first disk d1 ₀ ing. However, this geometry may also be different based on the geometry of the defined area surrounding the set user reference point 410. Similarly, in the example shown in FIGS. 6A-6E, the area surrounding anchor point 420 is defined as a circular disc. However, the shape of this area surrounding the anchor point 420 is also defined by the closed curve shape of the area surrounding the set controller reference point and the extension to the intersection of the geometry with the plane of the virtual feature 450.

  As shown in FIG. 7A, in some implementations, one or more virtual photospheres 570 (eg, first, second, and third virtual photospheres 570A, 570B, 570C) are in a virtual environment. It may be available to the user for selection. Each photosphere may provide a 360 degree panoramic experience of, for example, a particular feature, location, etc. in a virtual environment. To move to one of the virtual photosphere 570, the user may, for example, move the virtual beam 500 generated by the handheld electronic device 102 as described above, for example, the virtual photosphere 570, such as the third virtual photosphere 570C. Towards a selected one of them, as shown in FIG. 7A, one may move to a 360 degree panoramic experience inside virtual feature 550. Further manipulation of the handheld electronic device 102, for example, releasing the button pointing the virtual beam 550 to the selected virtual photosphere 570C, causes the user to select the selected virtual photosphere 570C as shown in FIG. 7B. It may be moved, ie teleported or transported.

  When moving, transporting or teleporting to the selected virtual photosphere 570, the user may choose to adjust the scale for the features in the virtual environment, as discussed in detail above. In particular, when navigating to the selected virtual photosphere 570, the user may, as described in detail above, the virtual elements included in the 360 degree panoramic experience provided by the selected virtual photosphere 570. , The size / scale may increase or the size / scale may decrease.

  Once inside the virtual photosphere 570, the user may move within the virtual photosphere 570. For example, the user may turn to view different portions of the 360 degree panoramic experience provided by virtual photosphere 570 and / or the user may view 360 degree panoramic provided by virtual photosphere 570 The virtual location C may be walked from virtual location C to virtual location D, as shown in FIG. In some implementations, a user may walk, for example, within virtual photosphere 570 to a point that may be considered as the end of a 360 degree panoramic virtual display within virtual photosphere 570. The virtual elements displayed in front of the user, displayed in the 360 degree panoramic experience of the virtual photosphere 570, appear larger as the user walks or approaches in the direction of the virtual elements. Similarly, if the user looks back, for example, turns 180 degrees after arriving at virtual location D, he was once behind the user because he walked away from the virtual element displayed in that portion of virtual photosphere 570 The virtual element may appear small.

  A method 700 of navigating and / or scaling in an augmented and / or virtual reality environment according to the embodiments described herein is illustrated in FIG. Upon detecting the selection of a virtual feature (block 710), it may be determined whether a request to set a virtual anchor point has been received (block 720). The request to set the virtual anchor point may be input by the user, for example, as described in detail above with respect to FIGS. 4A and 6A. If a request to set a virtual anchor point is received, then a virtual anchor point may be set (block 730). If a virtual anchor point is set and a request to adjust the user's position, view, perspective, or scale with respect to the virtual environment (or a request to scale the virtual environment for the user) is received (block 750), the system May adjust the position and / or the view and / or the perspective and / or the scale as described in detail above with respect to FIGS. 4B-4F and 6B-6F according to the received request (block 760) ). If a virtual anchor point is not set, but a request is received to move or otherwise adjust the user's position relative to the selected virtual feature (block 740), the system requests the requested adjustment, It may execute according to the received request (block 760). Once the requested virtual reconciliation is complete (block 770), the process may be repeated until the virtual experience is over (block 780).

  FIG. 9 shows an example of a generic computing device 800 and a generic mobile computing device 850, which can be used with the techniques described herein. Computing device 800 may represent various forms of digital computers, such as laptops, desktops, tablets, workstations, personal digital assistants, televisions, servers, blade servers, mainframes, and other suitable computing devices. Intended. Computing device 850 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, and other similar computing devices. The components set forth herein, their connections and relationships, and their functions are merely exemplary and limit the implementation of the invention described and / or claimed herein. It is not a thing.

  The computing device 800 includes a processor 802, a memory 804, a storage device 806, a high speed interface 808 connected to the memory 804 and a high speed expansion port 810, and a low speed interface connected to a low speed bus 814 and the storage device 806. And 812. Processor 802 may be a semiconductor based processor. Memory 804 may be a semiconductor based memory. Each of the components 802, 804, 806, 808, 810, and 812 are interconnected using various busses and may be suitably mounted on a common motherboard or otherwise. Processor 802 includes instructions stored in memory 804 or storage 806 for displaying graphical information regarding the GUI on an external input / output device, such as a display 816 coupled to high speed interface 808. Instructions of execution within the device 800 can be processed. In other implementations, multiple processors and / or multiple buses may be used with multiple memories and memory types, as appropriate. Also, multiple computing devices 800 (e.g., as server banks, groups of blade servers, or multiprocessor systems) may be connected with each device providing a portion of the required operations.

  Memory 804 stores information in computing device 800. In one implementation, memory 804 is a volatile memory unit or units. In another implementation, memory 804 is a non-volatile memory unit or unit. Memory 804 may also be another form of computer readable medium, such as a magnetic or optical disk.

  Storage device 806 may provide computing device 800 with high capacity storage. In one implementation, the storage device 806 is a computer readable medium (eg, floppy disk device, hard disk device, optical disk device, or tape device), flash memory or other similar solid state memory device, or storage area It may be or be an array of devices including devices of a network or other configuration. The computer program product may be tangibly embodied in an information carrier. The computer program product may also include instructions that, when executed, perform one or more methods, as described above. The information carrier is a computer readable or machine readable medium, such as memory on memory 804, storage 806 or processor 802.

  High speed controller 808 manages bandwidth-intensive operations for computing device 800, and low speed controller 812 manages lower bandwidth-intensive operations. . Such allocation of functionality is merely exemplary. In one implementation, the high speed controller 808 has a memory 804 (eg, via a graphics processor or accelerator), a display 816, and a high speed expansion port 810 that can accept various expansion cards (not shown). Combined. In this implementation, slow controller 812 is coupled to storage 806 and slow expansion port 814. Low-speed expansion ports include various communication ports (eg, USB, Bluetooth, Ethernet, Wireless Ethernet), which may be, for example, a keyboard, pointing device, scanner, or switch or It may be coupled to one or more input / output devices, such as a network device such as a router, for example via a network adapter.

  The computing device 800 may be implemented in several different forms, as shown. For example, computing device 800 may be implemented as a standard server 820, or in a group of such servers. Additionally, computing device 800 may also be implemented as part of rack server system 824. Further, computing device 800 may be implemented on a personal computer such as laptop computer 822. Alternatively, the components of computing device 800 may be combined with other components of a mobile device (not shown), such as device 850. Each such device may include one or more of the computing devices 800, 850, and the overall system may be comprised of multiple computing devices 800, 850 in communication with one another.

  Computing device 850 includes, among other components, processor 852, memory 864, input / output devices such as display 854, communication interface 866, and transceiver 868. Device 850 may also include storage devices such as microdrives or other devices to provide additional storage. Each of components 850, 852, 864, 854, 866, and 868 are interconnected using various buses, and some of the components are, as appropriate, on a common motherboard or other It can be attached by the method.

  Processor 852 can execute instructions in computing device 850, including instructions stored in memory 864. The processor can be implemented as a chipset of chips that include separate analog and digital processors. The processor may, for example, coordinate other components of the device 850, such as a user interface, applications executed by the device 850, and control of wireless communications by the device 850.

  Processor 852 can communicate with the user via control interface 858 and display interface 856 coupled to display 854. Display 854 may be, for example, a TFT LCD display (thin film transistor liquid crystal display) or an OLED (organic light emitting diode) display, or other suitable display technology. Display interface 856 may comprise suitable circuitry for driving display 854 to present graphical and other information to the user. Control interface 858 can receive commands from the user and translate them for submission to processor 852. In addition, an external interface 862 may be provided in communication with the processor 852 to allow near field communication with other devices of the device 850. External interface 862 may, for example, provide wired communication in some implementations, or wireless communication in other implementations, and may use multiple interfaces.

  Memory 864 stores information in computing device 850. Memory 864 may be implemented as one or more computer readable media, one or more volatile memory units, or one or more non-volatile memory units. Also, an expansion memory 874 may be provided and connected to the device 850 via an expansion interface 472 which may include, for example, a SIMM (Single In Line Memory Module) card interface. Such expanded memory 874 may provide the device 850 with additional storage space, or may store application or other information in the device 850. Specifically, the expansion memory 874 can include instructions for performing or supplementing the above-described processing, and can also include secure information. Thus, for example, the expanded memory 874 can be provided as a security module of the device 850, and the expanded memory 874 can be programmed with instructions that allow the secure use of the device 850. In addition, secure applications can be provided along with additional information via the SIMM card, such as placing identification information on the SIMM card in a manner that is not capable of hackers.

  The memory may include, for example, flash memory and / or NVRAM memory, as described below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product comprises instructions that, when executed, perform one or more methods, as described above. The information carrier is a computer readable or machine readable medium such as memory 864, expanded memory 874 or memory on processor 852 which may be received, for example, through transceiver 868 or external interface 862.

  Device 850 may communicate wirelessly via communication interface 866, which may include digital signal processing circuitry, as desired. Communication interface 866 is based on various modes or protocols, such as GSM voice call, SMS messaging, EMS messaging or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. It can communicate. Such communication may occur, for example, via a radio frequency transceiver 868. In addition, short-range communication can be performed using Bluetooth, WiFi, or other such transceivers (not shown) or the like. In addition, a GPS (Global Positioning System) receiver module 870 can provide additional navigation related wireless data and location related wireless data to the device 850, which information is executed on the device 850. It can be used as needed by the application.

  Also, device 850 may communicate audible using voice codec 860. Audio codec 860 can receive spoken information from the user and convert it into usable digital information. The audio codec 860 can also generate audible sounds that the user can hear, for example, via speakers in the handset of the device 850, etc. Such audio may include audio from voice telephone calls, may include recorded audio (eg, voice messages, music files, etc.), and may further include audio generated by an application operating on device 850. Can be included.

  Computing device 850 may be implemented in several different forms, as shown. For example, computing device 850 may be implemented as a cellular telephone 880. The computing device 850 may be implemented as part of a smartphone 882, a personal digital assistant, or other similar mobile device.

  Various implementations of the systems and techniques described herein may be in digital electronic circuits, integrated circuits, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. It is possible to realize. These various implementations may be dedicated or general purpose coupled to the storage system, one or more input devices, and one or more output devices to transmit and receive data and instructions. It is possible to include an implementation in one or more computer programs executable and / or interpretable on a programmable system including a processor.

  These computer programs (also known as programs, software, software applications or codes) comprise machine language instructions for a programmable processor, in a high level procedural language and / or object oriented programming language, and / or assembly / machine language It is possible to implement in As used herein, the terms "machine-readable medium" and "computer-readable medium" provide machine-readable instructions and / or data to a programmable processor, including machine-readable media that receive machine-language instructions as a machine-readable signal. Computer program product, apparatus, and / or apparatus (eg, magnetic disk, optical disk, memory, programmable logic device (PLD)) used to The term "machine readable signal" refers to any signal used to provide machine language instructions and / or data to a programmable processor.

  In order to interact with the user, the systems and techniques described herein include a display (e.g., a CRT (cathode ray tube) monitor or LCD (liquid crystal display) monitor) for displaying information to the user; May be implemented on a computer having a keyboard and a pointing device (eg, a mouse or a trackball) by which it can input to the computer. Other types of devices may be used as well to interact with the user, for example, the feedback provided to the user may be sensory feedback (eg, visual feedback, auditory feedback, or haptics) Feedback), and the input from the user can be received in any form including acoustic input, voice input, or tactile input.

  The systems and techniques described herein may include back-end components (eg, as data servers), middleware components (eg, application servers), or front-end components (eg, users). A computing system, or any such back-end component, middleware component, or front-end component, including a client computer having a graphical user interface or web browser through which implementations of the described systems and techniques can interact. It is possible to implement in combination. The components of the system can be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

  The computing system can include clients and servers. The client and server are generally separate from one another and typically interact through a communication network. The relationship of client and server results from computer programs running on each computer and having a client-server relationship to each other.

Several embodiments have been described. Nevertheless, it will be understood that various modifications may be made without departing from the scope and spirit of the invention.
In addition, the logic flow depicted in the figures does not require the particular order or order shown to achieve desirable results. In addition, other steps may be provided, or steps may be omitted from the described flow. Also, other components may be added or omitted to the described system. Accordingly, other embodiments are within the scope of the following claims.

  Implementations of the various techniques described herein may be performed in digital electronic circuitry, computer hardware, firmware, software, or any combination thereof. The implementation may be tangible in an information carrier, for example a machine readable storage device (computer readable medium), or for processing by a data processor such as a programmable processor, a computer, or a plurality of computers, or for controlling the operation thereof. A computer program product embodied in a computer program may be implemented as a computer program product. Thus, a computer readable storage medium may be configured to store instructions which, when executed, cause a processor (e.g. a processor in a host device, a processor in a client device) to perform a method.

  A computer program, such as the computer program described above, may be written in any form of a programming language, including a compiled or interpreted language, suitable as a stand-alone program or module, for use in a component, subroutine, or computing environment It may be deployed in any form, including other units. The computer program may be deployed and processed on one computer or a plurality of computers in one place, or may be distributed in a plurality of places interconnected by a communication network.

  The steps of the method may be performed by one or more programmable processors executing a computer program that performs functions by processing input data and generating output. The method steps may also be performed by dedicated logic circuits such as FPGAs (field programmable gate arrays) or ASICs (application specific integrated circuits), and the devices may be implemented as those circuits.

  Processors suitable for the processing of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. Elements of a computer may include one or more processors for executing instructions and one or more memory devices for storing instructions and data. In general, the computer may also include or operate one or more mass storage devices for data storage, such as magnetic, magneto-optical or optical disks, to receive and / or transmit data. It may be connected as possible. Information carriers suitable for implementing computer program instructions and data include, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory, magnetic disks such as internal hard disks or removable disks, and magneto-optical disks. Includes all forms of non-volatile memory, including CD-ROM and DVD-ROM discs. The processor and the memory may be added to or incorporated in dedicated logic circuits.

  In order to provide interaction with the user, the implementation may be a display device such as a cathode ray tube (CRT), a light emitting diode (LED), or a liquid crystal display (LCD) monitor for displaying information to the user, and the user thereby computing On a computer having a keyboard and pointing device, such as a mouse or a trackball, capable of providing input. Other types of devices may be used as well to interact with the user, for example, the feedback provided to the user may be sensory feedback, such as any of visual feedback, auditory feedback, or tactile feedback. It may be in form and the input from the user may be received in any form including acoustic input, voice input or tactile input.

  An implementation may include, for example, a back end component as a data server, a middleware component, such as an application server, or a front end component, for example a client having a graphical user interface or web browser through which a user can interact with the implementation It may be performed by a computing system, including a computer, or any combination of such back end components, middleware components, or front end components. The components can be interconnected by digital data communication, eg, any form or medium of communication network. Examples of communication networks include local area networks (LANs) and wide area networks (WANs) such as the Internet.

Further implementations are summarized in the following example.
Example 1: A method comprising: creating a virtual environment within a display of a head mounted display device; detecting a first input at a user interface of a controller in communication with the head mounted display device; Setting an anchor point to the selected virtual feature in the virtual environment in response to the setting of the anchor point, the step of detecting the second input, and the anchor point associated with the second input. The process of defining an area in a virtual environment including virtual features, and maintaining a portion of the virtual features within the defined area within the user view of the virtual environment while maintaining the position and scale of the virtual features in the virtual environment Adjusting at least one of the adjusting steps.

  Example 2: The anchor point setting step detects an intersection point between a virtual target line and a virtual feature, detects an intersection point, sets an anchor point at the detected intersection point between the virtual target line and the virtual feature, and Defining an area in a virtual environment including a virtual feature associated with an anchor point based on the set position of the controller reference point with respect to the set user reference point; The method described in.

  Example 3: The method according to example 2, wherein the configured user reference point corresponds to a fixed point on the user head mounted display device and the configured controller reference point corresponds to a fixed point to the controller.

  Example 4: The intersection point detection step is a step of defining a virtual target line, where the virtual target line starts from the set user reference point, and the virtual target line is set from the start point at the set user reference point The method of example 2 or 3 including the steps of extending through the controller reference point.

Example 5: The method according to any one of Examples 1 to 4, wherein the set controller reference point defines the center of the virtual closed curve.
Example 6: A location reference area definition step defines a first virtual disk defining a first virtual disk having a center defined by a configured controller reference point, a configured user reference point and a first virtual disk Defining a virtual cone based on the virtual cone, wherein the virtual cone extends toward the virtual feature starting from the set user reference point and contacts the first virtual disk, a virtual cone and a virtual cone and a virtual Defining a second virtual disk at the intersection with the feature, and a second virtual disk defining step.

  Example 7: The first virtual disk defining step is a step of defining a first radius of the first virtual disk, wherein the first radius extends from the set controller reference point to the circumference of the first virtual disk And the second virtual disk defining step is a step of defining the center of the second virtual disk, wherein the center of the second virtual disk corresponds to the anchor point, and the step of, and the circumference of the second virtual disk Defining, wherein the circumference corresponds to the intersection of the cone and the plane of the virtual feature, and defining the second radius of the second virtual disk, the second radius being And e. Extending from the center of the second virtual disk to the circumference of the second virtual disk.

Example 8: The method according to Example 6 or 7, wherein the virtual cone definition step includes the step of defining a half apex angle between the virtual target line and the outer circumference of the virtual cone.
Example 9: When the configured controller reference point moves relative to the configured user reference point, the first radius of the first virtual disk and the second virtual radius of the second virtual disk each remain constant, The method according to Example 7 or 8, wherein the angle of the half apex angle changes when the set controller reference point moves with respect to the set user reference point.

  Example 10: The step of receiving the second input includes the step of detecting the movement of the set controller reference point relative to the set user reference point, and the adjusting step is a user reference in which the set controller reference point is set. The step of maintaining the anchor point on the virtual target line when moving relative to the point, and when the set controller reference point moves relative to the set user reference point, the first radius of the first virtual disk is fixed. And the second controller is configured to move the set controller reference point relative to the set user reference point, and adjust at least one of the position and the scale of the virtual feature, the second virtual disk second The steps of maintaining the diameter constant with respect to the virtual environment, moving the set controller reference point relative to the set user reference point, and at least one of the position and the scale of the virtual feature is adjusted Adjusting the apex angle, and the method according to any one of Examples 1 to 9, comprising:

  Example 11: The step of receiving the second input includes the step of detecting the movement of the set controller reference point relative to the set user reference point, and the adjusting step is a user reference in which the set controller reference point is set. Maintaining the anchor point on the virtual target line when moving relative to the point, and moving the anchor point, and moving part of the virtual feature in the defined area surrounding the anchor point when the anchor point moves; Maintaining within a defined area, and adjusting at least one of position and scale of a portion of the virtual feature in the defined area surrounding the anchor point as the anchor point moves Examples 1 to 10 The method according to one example Zureka.

  Example 12: A defined area of a virtual feature surrounding an anchor point corresponds to a second virtual disk and when at least one of the position and the scale of the virtual feature is adjusted in response to movement of the anchor point, The method of any one of examples 6-11, wherein the virtual radius of the second virtual disk remains constant with respect to the virtual environment.

  Example 13: A system comprising a computing device configured to create an immersive virtual environment, the computing device including: a memory for storing executable instructions; and a processor configured to execute the instructions. And the processor, in the computer device, generating a virtual environment, detecting a first input in a user interface of a controller in communication with the head mounted display device, and responding to the first input as virtual Setting an anchor point to the selected virtual feature in the environment, setting an anchor point, detecting a second input, and responding to the second input, arranging the area of the feature within the area of the anchor Defining in the defined area within the user view of the virtual environment. While maintaining a portion of Icha system to execute, and at least one adjusting, adjustment process of the position and scale of a virtual feature in a virtual environment.

  Example 14: In the anchor point setting step, the processor detects the intersection of the virtual target line and the virtual feature, and the virtual target line is set to correspond to a fixed point on the user head mounted display Setting the anchor point at the detected intersection of the virtual target line and the virtual feature, defined based on the starting point at the user reference point and extending through the set controller reference point corresponding to the fixed point to the controller Defining the area of the feature, surrounding the anchor point, based on the position of the set controller reference point relative to the set user reference point; The system according to 3.

  Example 15: In the position reference area defining step, the processor defines a first virtual disk at a set user controller point, and defines a virtual cone based on the set user reference point and the first virtual disk. A virtual cone extending toward the virtual feature starting from the set user reference point and contacting the first virtual disk, at a virtual cone defining step, and at a crossing point of the virtual cone and the virtual feature Defining a virtual disk, the system according to example 13 or 14 configured to perform a second virtual disk definition step.

  Example 16: In the process of defining a first virtual disk as a virtual crosshair, the processor is a process of defining the center of the first virtual disk, and the center of the first virtual disk corresponds to a set controller reference point And defining a first radius of the first virtual disk, the first radius extending from the center of the first virtual disk at the set controller reference point to the circumference of the first virtual disk The system according to example 15, configured to perform a process.

  Example 17: In the second virtual disk defining step, the processor defines the center of the second virtual disk, and the center of the second virtual disk corresponds to the anchor point, and the step of the second virtual disk Defining a circumference, the circumference corresponding to the intersection of the cone and the virtual feature, and defining a second radius of the second virtual disk, the second radius being Example configured to perform the steps of: extending from the center of the second virtual disk to the circumference of the second virtual disk; and defining the semi-apex angle between the virtual target line and the outer circumference of the virtual cone The system according to 15 or 16.

  Example 18: In the adjusting step, the processor detects movement of the set controller reference point with respect to the set user reference point as a second input, and the set controller reference point is set to the user reference point The process of maintaining the anchor point on the virtual target line when moving relative to the user, and maintaining the first radius of the first virtual disk constant when moving the set controller reference point relative to the set user reference point And when the set controller reference point is moved relative to the set user reference point and at least one of the position and the scale of the virtual feature is adjusted, the second virtual disk Maintaining the second radius constant with respect to the virtual environment, moving the second controller reference point relative to the set user reference point, and / or at least one of the position and the scale of the virtual feature The system according to any one of the examples 13-17, configured to perform the step of adjusting the half apex angle when A is adjusted.

  Example 19: In the second radius maintenance step, the processor maintains, within the defined area, a portion of virtual features in the defined area surrounding the anchor point as the anchor point moves. Adjusting at least one of the position and the scale of a portion of the virtual feature in the defined area surrounding the anchor point when the anchor point moves with adjustment of at least one of the position and the scale of The system according to example 18, configured to:

  Example 20: A defined area of a virtual feature surrounding an anchor point corresponds to a second virtual disk and when at least one of the position and the scale of the virtual feature is adjusted in response to movement of the anchor point, The system according to any one of examples 15 to 19, wherein the virtual radius of the second virtual disk remains constant with respect to the virtual environment.

  While certain features of the described implementations have been described as described herein, several variations, substitutions, modifications, and equivalents will be apparent to those of ordinary skill in the art. It is therefore to be understood that the appended claims are intended to cover all such variations, and that variations are included within the scope of the implementation. It should be understood that they have been presented by way of example only and not as various modifications in form and detail. Any portion of the devices and / or methods described herein may be combined in any combination except mutually exclusive combinations. The implementations described herein may include various combinations and / or functions, subcombinations of components, and / or features of the different implementations described.

Claims (20)

Method,
Creating a virtual environment within the display of the head mounted display device;
Detecting a first input received at a user interface of a handheld controller in communication with the head mounted display device;
Setting an anchor point to the selected virtual feature in the virtual environment in response to the first input;
Detecting a second input;
In response to the second input,
Defining an area in the virtual environment including the virtual feature associated with the anchor point based on the set position of the controller reference point with respect to the set user reference point, the set user The reference point corresponds to a fixed point for the head mounted display device, the set controller reference point corresponds to a fixed point for the handheld controller, the set user reference point and the set controller reference point Said step being on an imaginary line with said anchor point ,
Wherein in the user field of view of the virtual environment, while maintaining the area A defined, for adjusting at least one of the position and scale of the virtual feature in the virtual environment, the method comprising the adjusting step. In the anchor point setting process,
An intersection point detection step of detecting an intersection point of a virtual target line and the virtual feature;
Wherein on the detected intersection of the virtual target line and the virtual feature, The method of claim 1 including the steps of setting the anchor point. The set user reference point corresponds to a fixed point on the head-mounted display device, the set controller reference point A method according to claim 2 which corresponds to a fixed point with respect to the handheld the controller.   The intersection location detection step is a step of defining the virtual target line, and the virtual target line starts from the set user reference point, and the virtual target line corresponds to the set user reference point. 4. The method of claim 3, including the step of extending from the start point through the set controller reference point.   5. The method of claim 4, wherein the set controller reference point defines a center of a virtual closed curve. The step of defining the area in the virtual environment includes
A first virtual disk definition step of defining a first virtual disk having a center defined by the set controller reference point;
Defining a virtual cone based on the set user reference point and the first virtual disk, wherein the virtual cone extends toward the virtual feature starting from the set user reference point; 1 virtual cone definition process in contact with a virtual disk;
The method of claim 1 , further comprising the step of: defining a second virtual disk at an intersection of the virtual cone and the virtual feature. The first virtual disk defining step is a step of defining a first radius of the first virtual disk, and the first radius extends from the set controller reference point to the circumference of the first virtual disk. , And the second virtual disk defining step includes
Defining a center of the second virtual disk, wherein the center of the second virtual disk corresponds to the anchor point;
Defining a circumference of the second virtual disk, the circumference corresponding to an intersection of the cone and a virtual plane located at the anchor point, the virtual plane being perpendicular to the virtual target line There is a process
Defining a second radius of the second virtual disk, wherein the second radius extends from the center of the second virtual disk to the circumference of the second virtual disk. The method according to Item 6.   The method according to claim 7, wherein the step of defining the virtual cone includes the step of defining a half apex angle between the virtual target line and the outer periphery of the virtual cone. When the configured controller reference point moves relative to the configured user reference point, the first radius of the first virtual disk remains constant for the user, and the second radius of the second virtual disk is The second radius of the virtual remains constant with respect to the virtual environment, and the angle of the semi-apical angle changes when the set controller reference point moves relative to the set user reference point The method according to item 8. The second step of receiving the input includes the step of detecting the movement of the set controller reference point for the configured user reference point, the adjustment step,
Maintaining the anchor point on the virtual target line as the set controller reference point moves relative to the set user reference point;
Maintaining the first radius of the first virtual disk constant for the user when the set controller reference point moves relative to the set user reference point;
When the set controller reference point moves relative to the set user reference point and at least one of the position of the virtual feature and the scale is adjusted, the second of the second virtual disk a step of maintaining a constant for a second radius on said virtual environment,
Adjusting the half apex angle when the set controller reference point moves relative to the set user reference point and at least one of the position of the virtual feature and the scale is adjusted The method according to claim 8, comprising The second step of receiving the input includes the step of detecting the movement of the set controller reference point for the configured user reference point, the adjustment step,
Maintaining the anchor point on the virtual target line to move the anchor point when the set controller reference point moves relative to the set user reference point;
Maintaining the portion of the virtual feature in the defined area surrounding the anchor point within the defined area as the anchor point moves;
Adjusting at least one of the position of the portion of the virtual feature and the scale in the defined area surrounding the anchor point as the anchor point moves. the method of.   The defined area of the virtual feature surrounding the anchor point corresponds to the second virtual disk, and at least one of the position of the virtual feature and the scale is responsive to movement of the anchor point The method according to claim 11, wherein when adjusted, the virtual radius of the second virtual disk remains constant with respect to the virtual environment. A system,
A computer device configured to create an immersive virtual environment, the computer device comprising
A memory for storing executable instructions;
A processor configured to execute the instructions;
Creating a virtual environment in the display of the head mounted display device ;
And detecting a first input received at the handheld controller of the user interface to communicate with the head-mounted display device,
Setting an anchor point to the selected virtual feature in the virtual environment in response to the first input;
Detecting a second input;
In response to the second input,
Defining the area of the feature such that the anchor point is placed in the area based on the set position of the controller reference point relative to the set user reference point, the set user reference The point corresponds to a fixed point to the head mounted display device, the set controller reference point corresponds to a fixed point to the controller of the handheld, and the set user reference point and the set controller reference point An anchor point is on an imaginary line ,
Wherein in the user field of view of the virtual environment, the system to be executed while maintaining a defined the area A, adjusts at least one of the position and scale of the virtual feature in the virtual environment, and adjusting step. In the anchor point setting step, the processor
Detecting an intersection between a virtual target line and the virtual feature, wherein the virtual target line is defined based on a start point at a set user reference point corresponding to a fixed point on a user head mounted display; Extending through a set controller reference point corresponding to a fixed point to the controller;
A system according to configured claim 13 to perform the steps of the anchor points, set on the detected intersections between the virtual target line and the virtual feature. In the step of defining the area of the feature surrounding the anchor point, the processor
A step of defining the first virtual disk, to the set controller reference point,
Defining a virtual cone based on the set user reference point and the first virtual disk, wherein the virtual cone extends toward the virtual feature starting from the set user reference point; 1 virtual cone definition process in contact with a virtual disk;
15. A system according to claim 14, configured to perform a second virtual disk definition step, defining a second virtual disk at the intersection of the virtual cone and the virtual feature. In the step of defining the first virtual disk as a virtual cross hair, the processor is configured to:
Defining a center of the first virtual disk, wherein the center of the first virtual disk corresponds to the set controller reference point;
Defining a first radius of the first virtual disk, wherein the first radius extends from the center of the first virtual disk at the set controller reference point to the circumference of the first virtual disk 20. The system of claim 15, configured to perform: a. In the second virtual disk definition step, the processor
Defining a center of the second virtual disk, wherein the center of the second virtual disk corresponds to the anchor point;
Defining a circumference of the second virtual disk, the circumference corresponding to an intersection of the cone and the virtual feature;
Defining a second radius of the second virtual disk, the second radius extending from the center of the second virtual disk to the circumference of the second virtual disk;
17. A system according to claim 16, wherein: defining a half apex angle between the virtual target line and the outer circumference of the virtual cone. In the adjusting step, the processor
Detecting movement of the set controller reference point with respect to the set user reference point as a second input;
Maintaining the anchor point on the virtual target line as the set controller reference point moves relative to the set user reference point;
Maintaining the first radius of the first virtual disk constant for the user when the set controller reference point moves relative to the set user reference point;
When the set controller reference point moves relative to the set user reference point and at least one of the position of the virtual feature and the scale is adjusted, the second of the second virtual disk the second radius is maintained constant for the virtual environment, and a second radius sustain step,
Adjusting the half apex angle when the set controller reference point moves relative to the set user reference point and at least one of the position of the virtual feature and the scale is adjusted The system according to claim 17, configured to perform and. Maintaining the second radius of the second virtual disk constant with respect to the virtual environment when the set controller reference point moves relative to the set user reference point; Is
Maintaining the portion of the virtual feature in the defined area surrounding the anchor point within the defined area as the anchor point moves;
The position and the scale of the portion of the virtual feature in the defined area surrounding the anchor point when the anchor point moves with adjustment of at least one of the position of the virtual feature and the scale; 20. A system according to claim 17 , configured to perform at least one of adjusting.   The defined area of the virtual feature surrounding the anchor point corresponds to the second virtual disk, and at least one of the position of the virtual feature and the scale is responsive to movement of the anchor point 20. The system of claim 19, wherein when adjusted, the virtual radius of the second virtual disk remains constant with respect to the virtual environment.