Classifications

• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
  • G06N20/00—Machine learning
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
  • G09G5/14—Display of multiple viewports
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/16—Sound input; Sound output
  • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• • G—PHYSICS
  • G06—COMPUTING OR CALCULATING; COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
  • G06T19/006—Mixed reality

Definitions

• the present disclosure relates to modal control of a virtual reality (VR) device, and more particularly, to modal control of a VR device based on hand locations.
• VR virtual reality
• VR devices which use a graphics processing unit (GPU) to render graphics from a computer device to a display device based on rendering instructions received from the computer device.
• GPU graphics processing unit
• a scene produced on a display device can be modified based on user input, where the input may include detected movement of a head (e.g., detected movement of the VR device, such as a head mounted display (HMD)) or a hand within a field of view of the VR device.
• HMD head mounted display
• a common problem in the application of VR devices is the use of multiple user experiences (UXs) on a display device.
• UXs user experiences
• a displayed view is based on a tabular (windowed) experience similar to that used by a desktop device.
• a tabular experience may include multiple windows within a displayed view and cursor positioning to perform window and application functions.
• a tabular experience may inhibit a user from fully engaging in the richness of a VR experience.
• simultaneous use of multiple windows within the displayed view may detract from interaction by a user with a primary UX due to congestion in the displayed view caused by multiple windows.
• the removal and addition of windows from/to the displayed view may disturb the interaction of the user with another window due to a change in position by a head of the user when moving a cursor position within a field of view of a VR device.
• the present disclosure relates to methods and devices for initiating application and system modal control of a device based on predicted locations of a hand outside of a field of view of the device.
• the computer device may include a memory storing data and instructions, a processor in communication with the memory and configured to execute the data and instructions, a display rendering a primary user experience (UX), and an operating system in communication with the memory and the processor.
• the operating system and processor may be operable to receive hand motion information from a hand positional tracking system that tracks a position of a hand of a user, determine the hand is outside a field of view of the display based on the hand motion information, predict, using machine learning, a location of the hand while outside the field of view, and interact with a secondary UX on the display based on the predicted location of the hand.
• Another example implementation relates to a method for modal control of a computer device.
• the method may include receiving hand motion information from a hand positional tracking system that tracks a position of a hand of a user, determining that the hand is outside a field of view of the computer device based on the hand motion information, predicting a location of the hand while outside the field of view, and rendering a secondary UX on a display of the computer device based on the predicted location of the hand.
• the computer-readable medium may include at least one instruction for causing the computer device to receive hand motion information from a hand positional tracking system that tracks a position of a hand of a user, at least one instruction for causing the computer device to determine that the hand is outside a field of view of the computer device based on the hand motion information, at least one instruction for causing the computer device to predict, using machine learning, a location of the hand while outside the field of view, and at least one instruction for causing the computer device to render a secondary UX on a display of the computer device based on the predicted location of the hand.
• FIG. 1 is a schematic block diagram of an example computer device in accordance with an implementation of the present disclosure
• FIG. 2 is an example of a field of view of a computer device in accordance with an implementation of the present disclosure
• FIG. 3 is an example method flow for modal control in a computer device in accordance with an implementation of the present disclosure.
• Fig. 4 is a schematic block diagram of an example device in accordance with an implementation of the present disclosure.
• a computer device may include a device that generates and/or displays virtual reality (VR) images (e.g., from at least one virtual environment input), mixed reality (MR) images (e.g., from at least two virtual environment inputs), and/or augmented reality (AR) images (e.g., from at least one virtual environment input and one real environment input).
• VR virtual reality
• MR mixed reality
• AR augmented reality
• Modal control is a technique whereby a primary user experience (UX) is subordinate to a secondary UX.
• a user may interact with the secondary UX while the primary UX is, for example, disabled or set to the background.
• Examples of a primary UX may include a primary window/image including a VR image, a MR image, a AR image, desktop view, or first application.
• Examples of a secondary UX may include a secondary window/image such as a second application, a utility window image for viewing application or system settings associated with a first application, a menu, or available command options.
• an interaction by a user with the primary UX and the secondary UX may be controlled based on the interaction of the user with the cursor via an input device such as a hand controller or a motion sensor for detected movement of a head mounted display (HMD).
• an input device such as a hand controller or a motion sensor for detected movement of a head mounted display (HMD).
• HMD head mounted display
• a tabular environment may include multiple windows within a displayed view and cursor positioning to perform window and application functions.
• a tabular experience may diminish a displayed view accessible to a user of an HMD, the user may be required to release focus from a primary UX in order to access a secondary UX, or a head of the user may need to be moved in order to move a cursor in the VR environment.
• the present disclosure provides a tubular environment within a VR environment that improves a user experience through the use of modal control.
• the present disclosure may place a secondary UX in a predefined three-dimensional (3D) location of a display.
• the secondary UX may be accessible to a user based on the placement of one or more hands of the user at a location that is outside of the field of view of the computer device.
• the secondary UX may become accessible when the computer device determines that the user moved one or more hands outside of the field of view.
• the one or more hands are no longer trackable by the computer device.
• the computer device uses Artificial Intelligence and/or Machine Learning to predict the location of the one or more hands when outside the field of view of the computer device.
• the computer device may activate the secondary UX such that the user may interact with the secondary UX using a voice controller or a hand controller such as a thumb pad.
• the user may interact with the secondary UX by returning a hand(s) to the field of view of the computer device.
• the computer device monitors where a user is looking and whether the one or more hands of the user is located in a field of view.
• the computer device may predict based on telemetry, where the one or more hands will be positioned outside of the field of view.
• a modal controller may be notified of the predicted location of the one or more hands and a secondary UX associated with the predicted location of the one or more hands may be displayed. Input from an external controller may then be received and used to interact with the secondary UX. Interaction with the secondary UX may continue until the user dismisses, minimizes, or closes the secondary UX via the external controller or, in some implementations, the one or more hands returns to the field of view of the computer device.
• whether a single hand or both hands are required to be outside of the field of view of the computer device may be based on an application using modal control.
• some applications using modal control may only rely on the use of one hand (e.g., right hand) while other applications may require both hands to perform the below described functions of a VR device.
• the remaining portions of the present disclosure refer to one or more hands as hand(s) unless there is a specific implementation describing specific use of one hand or both hands.
• an example system 100 for initiating modal control based on a predicted location 220 of a hand(s) 204 may include a computer device 102 in communication with a display device 106 via a network 17 and/or through a wired or wireless connection.
• the computer device 102 may execute an application 10 to perform modal control.
• the application 10 may be computer readable code or instructions executable to perform, for example, a game, an office productivity application, or any other type of application.
• the computer device 102 may generate and communicate image data for displaying one or more images (e.g., display frame 25) defining one or more scenes of the application 10 on the display device 106.
• Examples of the display device 106 may include an HMD device, and may have one or more display panels 32 (e.g., light emitting diode (LED), organic LED (OLED), or liquid crystal display (LCD)) capable of outputting an image for viewing by a user 202 of the system 100.
• display panels 32 e.g., light emitting diode (LED), organic LED (OLED), or liquid crystal display (LCD)
• the computer device 102 may communicate with a positional tracking system 104 via the network 17 and/or through a wired or wireless connection to obtain hand motion information 14 to apply to the application 10.
• the hand motion information may correspond to a position or a movement of the hand(s) 204 within the field of view 210 or the absence of the hand(s) from the field of view 210.
• the positional tracking system is configured to generate a field of view 210 for tracking a hand(s) 204 of the user 202.
• the positional tracking system 104 may include, but is not limited to, one or more sensors (e.g., cameras), emitters (e.g., IRLEDs), inertial devices (accelerometer and/or gyroscope), and/or any other position sensing system capable of detecting a position, an orientation, a movement, a direction of movement, and/or a speed of movement of the hand(s) 204 of the user 202 of the system 100 within a field of view 210 of the computer device 102.
• An example of the field of view 210 of the computer device 102 is shown by FIG. 2.
• the field of view 210 may be a sensing area of the sensors (e.g., cameras) of the positional tracking system 104.
• the field of view 210 may correspond to a displayed area of the display device 106 (e.g., HMD device).
• the field of view 210 may correspond to a real life view of the user 202 while using the display device 106.
• the field of view 210 may correspond to virtual reality view of the user 202 while using the display device 106.
• the positional tracking system 104 may include but is not limited to one or more depth cameras and one or more inertial measurement units (IMUs), tracking software, a simultaneous localization and mapping algorithm, and one or more processors.
• the positional tracking system 104 may include but is not limited to one or more sensors (e.g., cameras) for detecting one or more emitters (e.g., IR LEDs) located in the environment, and one or more processors.
• the positional tracking system 104 may be configured to separately track both hand(s) 204 of the user 202.
• the positional tracking system 104 may determine movement of a first hand (e.g., right hand) and determine whether the first hand is outside or inside the field of view 210 of the computer device 102 and/or determine movement of a second hand (e.g., left hand) and whether the second hand is outside or inside the field of view 210.
• a first hand e.g., right hand
• a second hand e.g., left hand
• the positional tracking system 104 may be located on the display device 106 (e.g., HMD) and may include cameras, IMUs, tracking devices, or any other device for tracking head movements of the user.
• the positional tracking system 104 may be configured to distinguish between the head movements (e.g., movement by head of user which causes hand(s) 204 to be outside field of view 210) of the user and hand movements (e.g., movement of hand(s) 204 to exit field of view 210) of the user when generating the hand motion information 14.
• the computer device 102 may also communicate with a hand prediction system 110 via the network 17 and/or through a wired or wireless connection to obtain hand prediction information 112 to apply to the application 10.
• the hand prediction system 110 is configured to predict a location of the hand(s) 204 of the user 202 when the hand(s) 204 is outside the field of view 210 of the computer device 102, as shown by FIG. 2. For example, when the hand(s) 204 of the user 202 is removed from the field of view 210, the position tracking system 104 is no longer able to determine a location of the hand(s) 204.
• the hand prediction system 110 therefore predicts a location (predicted location 220) of the hand(s) 204 when outside the field of view 210 of the computer device 102. Prediction of the location of the hand(s) 204 may be based on hand motion information 14. For example, the predication of the location of the hand(s) 204 may be based on the position, the orientation, the movement, the direction of movement, and/or the speed of movement of the hand(s) 204 when moving from inside the field of view 210 to outside the field of view 210. In some examples, the prediction of the location of the hand(s) 204 may exclude information relating to head movement.
• the hand prediction system 110 may be a machine learning system configured to predict the location of the hand(s) 204 when outside the field of view 210 and to generate the hand prediction information 112.
• the hand prediction system 110 may be trained based on machine learning models using supervised learning, unsupervised learning, or a combination of both. In training the machine learning model using supervised learning, the model may be presented with labeled inputs and their desired outputs so that the model is trained to learn a general rule that maps inputs to outputs.
• no labels may be given to the machine learning model such that the model determines the structure of the input, where the model can discover hidden patterns in the input and/or learn more about predicting certain features (e.g., if the model has already been through supervised learning).
• the computer device 102 may communicate with one or more external controllers 108 via the network 17 and/or through a wired or wireless connection to obtain interaction information 12 to apply to the application 10.
• the external controller 108 may include, but is not limited to, a gamepad, joystick, keyboard, mouse, voice controller, or any other input device that may provide interaction information 12 for a virtual environment, such as a game, a virtual environment, or an augmented environment associated with the application 10.
• the computer device 102 may include an operating system 115 executed by one or more processors 36 and/or a memory 38.
• the memory 38 of the computer device 102 may be configured for storing data and/or computer-executable instructions defining and/or associated with the operating system 115, and the processor 36 may execute such data and/or instructions to instantiate the operating system 115.
• An example of the memory 38 may include, but is not limited to, random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, or any type of memory usable by a computer, and/or any combination thereof.
• processor 36 may include, but is not limited to, any processor specially programmed as described herein, including a controller, microcontroller, application specific integrated circuit (ASIC), field programmable gate array (FPGA), system on chip (SoC), or other programmable logic or state machine.
• ASIC application specific integrated circuit
• FPGA field programmable gate array
• SoC system on chip
• the computer device 102 may include any mobile or fixed computer device, which may be connectable to the network 17.
• the computer device 102 may be, for example, a computer device such as a desktop or laptop or tablet computer, a cellular telephone, a gaming device, an MR device, a VR device, a music device, a television, a navigation system, a camera, a personal digital assistant (PDA), or a handheld device, or any other computer device having wired and/or wireless connection capability with one or more other devices and/or communication networks.
• a computer device such as a desktop or laptop or tablet computer, a cellular telephone, a gaming device, an MR device, a VR device, a music device, a television, a navigation system, a camera, a personal digital assistant (PDA), or a handheld device, or any other computer device having wired and/or wireless connection capability with one or more other devices and/or communication networks.
• PDA personal digital assistant
• the operating system 115 may include a graphics processing unit (GPU) 24 operable to render one or more image frames and determine when to transmit rendered image frames for presentation on the display device 106.
• the GPU 24 may receive data and/or instructions from one or more applications 10 and may generate one or more display frames (e.g., display frame 25) based on the received data and/or instructions.
• the GPU 24 may communicate with a display controller 26 to transmit rendered image frames (e.g., display frame 25) for presentation on the display device 106.
• operating system 115 may also include a display interface 28 that may be communicatively coupled with the processor 36 and/or the memory 38 for communicating with the display device 106 via a display port 30.
• the display controller 26 may instruct the display interface 28 to communicate a rendered image (e.g., display frame 25) to display on the display device 106.
• the display port 30 may transmit the rendered image to the display port 34.
• the display port 30 may include various types of ports including, but not limited to, high definition multimedia interface (HDMI) ports, display serial interface (DSI) ports, mobile industry processor interface (MIPI) DSI ports, universal serial bus (USB) ports, Firewire ports, or other embedded or external wired or wireless display ports that can allow communications, for example, via the network 17 between the computer device 102 and the display device 106.
• HDMI high definition multimedia interface
• DSI display serial interface
• MIPI mobile industry processor interface
• USB universal serial bus
• Firewire ports or other embedded or external wired or wireless display ports that can allow communications, for example, via the network 17 between the computer device 102 and the display device 106.
• the computer device 102 may also include the one or more applications 10 that may be executed by the processor 36 to present one or more images (e.g., display frame 25) on the display device 106.
• the one or more applications 10 executed by the processor 36 may also operate in cooperation with the operating system 115 and/or the GPU 24 to run, for example, a modal controller 20, and generate image frame(s) for presentation on the display device 106.
• the one or more applications 10 may include the modal controller 20 operable to render one or more display frames 25 on the display panel 32 based on hand motion information 14 and hand prediction information 112.
• the modal controller 20 may determine when to display a primary user experience (UX) 120 and/or a secondary UX 122.
• the primary UX 120 may be displayed when the hand(s) 204 of the user 202 is within the field of view 210 of the computer device 102
• the secondary UX 122 may be displayed when the hand(s) 204 is outside the field of view 210 of the computer 102.
• the secondary UX 122 may be displayed alone, simultaneous with the primary UX 120, or with another UX.
• the modal controller 20 determines whether the hand(s) 204 of the user 202 is inside or outside the field of view 210 of the computer device 102 based on hand motion information 14. Determining whether the hand(s) 204 is inside may be based on the sensors (e.g., cameras) of the positional tracking system 104 sensing the hand(s) 204 is within the field of view 210 of the computer device 102. In other words, the hand(s) 204 is inside of or within the field of view of the computer device 102 when the hand(s) 204 is within a tracking area of the sensors of the positional tracking system 104 (i.e., hand(s) 204 is trackable by the positional tracking system 104).
• the hand(s) 204 is outside of the field of view of the computer device 102 when the hand(s) 204 is not within a tracking area of the sensors of the positional tracking system 104 (i.e., hand(s) 204 is not trackable by the positional tracking system 104). Further, the modal controller 20 may determine whether the hand(s) 204 is inside or outside of the field of view 210 when the hand(s) 204 has remained outside the field of view 210 for longer than a predetermined period of time (e.g., 1 second). In an implementation, the predetermined period of time may be determined based on machine learning which may take into account habits, movements, and actions of the user 202.
• a predetermined period of time e.g. 1 second
• the modal controller 20 may also determine that the hand(s) 204 were moved out of the field of view 210 via hand movement.
• the modal controller 20 may determine a predicted location 220 of the hand(s) 204 based on the hand prediction information 112. The predicted location 220 may be used by the modal controller 20 to render the secondary UX for the display device 106. In some implementations, the secondary UX 122 is associated with the predicted location of the hand(s) 204.
• the secondary UX 122 may be a first utility UX (e.g., application settings), and when the predicted location is a second predicted location (e.g., hand(s) 204 of the user 202 outside of the field of view 210 and at waist level), the secondary UX 122 may be a second utility UX (e.g., system settings).
• first utility UX e.g., application settings
• second predicted location e.g., hand(s) 204 of the user 202 outside of the field of view 210 and at waist level
• the secondary UX 122 may be a second utility UX (e.g., system settings).
• the modal controller 20 may further be operable to receive input from the external controller(s) 108 or the hand(s) 204 to interact with the secondary UX 122.
• the input may be real world movements of the external controller(s) 108 or hand(s) 204.
• the modal controller 20 may receive the virtual motion information 12 from the external controller(s) 108 (e.g., hand controller) that corresponds to movement of a joystick and/or clicking of a button.
• the hand(s) 204 may return to the field of view 210, while the secondary UX 122 is being displayed, and the hand motion information 14 may correspond to a position or a movement of the hand(s) 204 within the field of view 210.
• Interaction with the secondary UX 122 may include adjusting a secondary window size (e.g., dismissing, minimizing, or maximizing a window), selecting an object (e.g., selecting button or text), scrolling through a menu, or performing any other type of interaction with the secondary UX 122.
• a secondary window size e.g., dismissing, minimizing, or maximizing a window
• selecting an object e.g., selecting button or text
• Scrolling through a menu e.g., text
• the positional tracking system 104 may separately track both hands 204 of the user 202 such that movements by a first hand (e.g., right hand) may be used by the modal controller 20 to interact with the secondary UX 122 (e.g., second application, settings), when the secondary UX 122 is rendered on the display device 106, and movements by a second hand (e.g., left hand) may be used by the modal controller 20 to interact with the primary UX 120 (e.g., operating system UX, first application, application launching, switching between applications, interaction with notifications).
• the modal controller 20 may transform the input (e.g., the virtual motion information 12 or the hand motion information 14) into virtual coordinates corresponding to interactions (e.g., cursor movement or button clicking) on the secondary UX 122 of the display frame 25.
• the computer device 102 While illustrated as being separate from each other, one skilled in the art would recognize that one or more of the computer device 102, the positional tracking system 104, the display device 106, the external controller 108, and/or the hand prediction system 110 may be remote from each other, located on each other, or partially remote from and partially located on each other.
• the method 300 may include receiving hand motion information from a hand positional tracking system that tracks a position of a hand(s) of a user.
• the computer device 102 may receive the hand motion information 14 from the positional tracking system 104.
• the hand motion information 14 may correspond to positional information of the hand(s) 204 of the user 202 while the hand(s) 204 is within the field of view 210 of the computer device 102 or the absence of the hand(s) from the field of view 210.
• the method 300 may include determining that the hand(s) is outside a field of view of a computer device based on the hand motion information. For example, as shown by FIGS. 1 and 2, the computer device 102 may determine that the hand(s) 204 is outside the field of view 210 when sensors of the positioning tracking system 104 do not detect the hand(s) 204.
• the method 300 may include predicting a location of the hand(s) while outside the field of view.
• the computer device 102 may receive hand prediction information 112 from the hand prediction system 110.
• the hand prediction information 112 may be based on machine learning which determines a predicted location 220 of the hand(s) 204.
• the hand prediction information 112 may further be based on stored historical actions performed by the user 202.
• the computer device 102 may store in the memory 38 any actions performed by the user 202.
• the saved actions may be used by the computer device to learn from tendencies and motions performed by the user 202 and to better predict the hand prediction information 112.
• the historical information may be shared with the hand prediction system 110.
• the method may include rendering a secondary UX on the display based on the predicted location of the hand(s). For example, as shown by FIGS. 1 and 2, the modal controller 20 may determine, based on the hand prediction information 112, a location of the hand(s) 204 while outside the field of view 210 of the computer device 102. Based on the predicted location 220, the modal controller 20 may instruct the display interface 28 to have the display panel 32 display the secondary UX 122.
• the method may optionally include determining that the hand(s) has returned to inside the field of view of the display based on the hand motion information. For example, as shown by FIGS. 1 and 2, the modal controller 20 may receive hand motion information 14 after the hand(s) 204 has returned to the field of view 210. In some implementations, when the modal controller 20 determines that the hand(s) 204 has returned to inside the field of view 210, the computer device 102 may dismiss, minimize or close the secondary UX 122. Alternatively, the modal controller 20 may receive commands for interacting with the secondary UX 122, as discussed in block 312 below.
• the method may optionally include receiving a command for interacting with the secondary UX.
• the modal controller 20 may receive a command from one or more of the external controller 108 or the positional tracking system 104.
• the command may correspond to commands from a hand controller or a voice controller to indicate an interaction with the secondary UX 122.
• the command may correspond to hand motions detected by the positional tracking system 104 after the hand(s) 204 has returned to the inside of the field of view 210.
• the method may optionally include interacting with the secondary UX based on the command.
• the modal controller 20 may interact with the secondary UX 122 based on the command received from the external controller 108 or the positional tracking system 104.
• interaction with the secondary UX 122 may include, for example, clicking on a button, highlighting images or text, or any other interaction with the secondary UX 122. Further, the interaction may be determined based on a predicted location of the hand(s) 204 of the user 202.
• the secondary UX 122 may be, for example, an application settings window associated with a first application or a second application, and when the hand(s) 204 is predicted to be a second location, the secondary UX 122 may be, for example, a systems settings window or a third application. Therefore, the interaction with the secondary UX 122 may be different based on the type of secondary UX 122 being displayed.
• FIG. 4 illustrated is an example of computer device 102 in accordance with an implementation, including additional component details as compared to FIG. 1.
• Computer device 102 may include a communications component 40 that provides for establishing and maintaining communications with one or more parties utilizing hardware, software, and services as described herein.
• Communications component 40 may carry communications between components on the computer device 102, as well as between the computer device 102 and external devices, such as devices located across a communications network and/or devices serially or locally connected to the computer device 102.
• communications component 40 may include one or more buses, and may further include transmit chain components and receive chain components associated with a transmitter and receiver, respectively, operable for interfacing with external devices.
• the computer device 102 may include a data store 42, which can be any suitable combination of hardware and/or software, that provides for mass storage of information, databases, and programs employed in connection with implementations described herein.
• the data store 42 may be a data repository for the applications 10 (FIG. 1), GPU 24 (FIG. 1), display controller 26 (FIG. 1), and/or display interface 28 (FIG. 1).
• the computer device 102 may also include a user interface component 44 operable to receive inputs from a user of computer device 102 and further operable to generate outputs for presentation to the user.
• the user interface component 44 may include one or more input devices, including but not limited to a keyboard, a number pad, a mouse, a touch-sensitive display, a navigation key, a function key, a microphone, a voice recognition component, any other mechanism capable of receiving an input from a user, or any combination thereof.
• the user interface component 44 may include one or more output devices, including but not limited to a display, a speaker, a haptic feedback mechanism, a printer, any other mechanism capable of presenting an output to a user, or any combination thereof.
• the user interface component 44 may transmit and/or receive messages corresponding to the operation of the applications 10, GPU 24, display controller 26, and/or display interface 28.
• the processor 36 may execute the applications 10, GPU 24, display controller 26, and/or display interface 28, and the memory 38 or data store 42 may store them.
• a component may be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer.
• an application running on a computer device and the computer device can be a component.
• One or more components can reside within a process and/or thread of execution and a component may be localized on one computer and/or distributed between two or more computers.
• these components can execute from various computer readable media having various data structures stored thereon.
• the components may communicate by way of local and/or remote processes such as in accordance with a signal having one or more data packets, such as data from one component interacting with another component in a local system, distributed system, and/or across a network such as the Internet with other systems by way of the signal.
• the term“or” is intended to mean an inclusive“or” rather than an exclusive“or.” That is, unless specified otherwise, or clear from the context, the phrase“X employs A or B” is intended to mean any of the natural inclusive permutations. That is, the phrase“X employs A or B” is satisfied by any of the following instances: X employs A; X employs B; or X employs both A and B.
• the articles“a” and“an” as used in this application and the appended claims should generally be construed to mean“one or more” unless specified otherwise or clear from the context to be directed to a singular form.
• DSP digital signal processor
• ASIC application specific integrated circuit
• FPGA field programmable gate array
• a general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine.
• a processor may also be implemented as a combination of computer devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Additionally, at least one processor may comprise one or more components operable to perform one or more of the steps and/or actions described above.
• a software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
• An exemplary storage medium may be coupled to the processor, such that the processor can read information from, and write information to, the storage medium.
• the storage medium may be integral to the processor.
• the processor and the storage medium may reside in an ASIC. Additionally, the ASIC may reside in a user terminal.
• processor and the storage medium may reside as discrete components in a user terminal. Additionally, in some implementations, the steps and/or actions of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a machine readable medium and/or computer readable medium, which may be incorporated into a computer program product.
• the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code on a computer- readable medium.
• Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another.
• a storage medium may be any available media that can be accessed by a computer.
• such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer.
• Disk and disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs usually reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer- readable media.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• General Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Computer Hardware Design (AREA)
• Software Systems (AREA)
• General Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Health & Medical Sciences (AREA)
• Computer Graphics (AREA)
• Multimedia (AREA)
• Artificial Intelligence (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Data Mining & Analysis (AREA)
• Evolutionary Computation (AREA)
• Medical Informatics (AREA)
• Computing Systems (AREA)
• Mathematical Physics (AREA)
• User Interface Of Digital Computer (AREA)

Priority Applications (3)

Applications Claiming Priority (4)

Publications (1)

Family

ID=68385186

Family Applications (1)

Country Status (5)

Cited By (1)

Families Citing this family (7)

Citations (2)

Family Cites Families (16)

• 2018
  • 2018-08-31 US US16/119,098 patent/US10860088B2/en active Active
• 2019
  • 2019-04-22 EP EP19727139.8A patent/EP3788460A1/en active Pending
  • 2019-04-22 KR KR1020207031557A patent/KR102732803B1/ko active Active
  • 2019-04-22 JP JP2020554878A patent/JP7252252B2/ja active Active
  • 2019-04-22 WO PCT/US2019/028599 patent/WO2019212793A1/en not_active Ceased

Patent Citations (2)

Also Published As

Similar Documents

Legal Events