WO2012118769A2 - Immersive display experience - Google Patents
Immersive display experience Download PDFInfo
- Publication number
- WO2012118769A2 WO2012118769A2 PCT/US2012/026823 US2012026823W WO2012118769A2 WO 2012118769 A2 WO2012118769 A2 WO 2012118769A2 US 2012026823 W US2012026823 W US 2012026823W WO 2012118769 A2 WO2012118769 A2 WO 2012118769A2
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- user
- data
- content
- display
- application
- Prior art date
Links
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/25—Output arrangements for video game devices
- A63F13/26—Output arrangements for video game devices having at least one additional display device, e.g. on the game controller or outside a game booth
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
- A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- A63F13/428—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
- A63F2300/1087—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
- A63F2300/1093—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera using visible light
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/30—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
- A63F2300/301—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device using an additional display connected to the game console, e.g. on the controller
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/30—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by output arrangements for receiving control signals generated by the game device
- A63F2300/308—Details of the user interface
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/60—Methods for processing data by generating or executing the game program
- A63F2300/6045—Methods for processing data by generating or executing the game program for mapping control signals received from the input arrangement into game commands
Definitions
- An immersive display environment is provided to a human user by projecting a peripheral image onto environmental surfaces around the user.
- the peripheral images serve as an extension to a primary image displayed on a primary display.
- FIG. 1 schematically shows an embodiment of an immersive display environment.
- FIG. 2 shows an example method of providing a user with an immersive display experience.
- FIG. 3 schematically shows an embodiment of a peripheral image displayed as an extension of a primary image.
- FIG. 4 schematically shows an example shielded region of a peripheral image, the shielded region shielding display of the peripheral image at the user position.
- FIG. 5 schematically shows the shielded region of FIG. 4 adjusted to track a movement of the user at a later time.
- FIG. 6 schematically shows an interactive computing system according to an embodiment of the present disclosure.
- Interactive media experiences are commonly delivered by a high quality, high resolution display.
- Such displays are typically the only source of visual content, so that the media experience is bounded by the bezel of the display.
- the user may perceive architectural and decorative features of the room the display is in via the user's peripheral vision.
- Such features are typically out of context with respect to the displayed image, muting the entertainment potential of the media experience.
- the ability to perceive motion and identify objects in the peripheral environment i.e., in a region outside of the high resolution display
- Various embodiments are described herein that provide the user with an immersive display experience by displaying a primary image on a primary display and a peripheral image that appears, to the user, to be an extension of the primary image.
- FIG. 1 schematically shows an embodiment of a display environment 100.
- Display environment 100 is depicted as a room configured for leisure and social activities in a user's home.
- display environment 100 includes furniture and walls, though it will be understood that various decorative elements and architectural fixtures not shown in FIG. 1 may also be present.
- a user 102 is playing a video game using an interactive computing system 110 (such as a gaming console) that outputs a primary image to primary display 104 and projects a peripheral image on environmental surfaces (e.g., walls, furniture, etc.) within display environment 100 via environmental display 116.
- an interactive computing system 110 such as a gaming console
- An embodiment of interactive computing system 110 will be described in more detail below with reference to FIG. 6.
- a primary image is displayed on primary display 104.
- primary display 104 is a flat panel display, though it will be appreciated that any suitable display may be used for primary display 104 without departing from the scope of the present disclosure.
- user 102 is focused on primary images displayed on primary display 104.
- user 102 may be engaged in attacking video game enemies that are shown on primary display 104.
- interactive computing system 110 is operatively connected with various peripheral devices.
- interactive computing system 110 is operatively connected with an environmental display 116, which is configured to display a peripheral image on environmental surfaces of the display environment.
- the peripheral image is configured to appear to be an extension of the primary image displayed on the primary display when viewed by the user.
- environmental display 116 may project images that have the same image context as the primary image.
- the user may be situationally aware of images and objects in the peripheral vision while being focused on the primary image.
- user 102 is focused on the wall displayed on primary display 104 but may be aware of an approaching video game enemy from the user's perception of the peripheral image displayed on environmental surface 112.
- the peripheral image is configured so that, to a user, the peripheral image appears to surround the user when projected by the environmental display.
- user 102 may turn around and observe an enemy sneaking up from behind.
- environmental display 116 is a projection display device configured to project a peripheral image in a 360- degree field around environmental display 116.
- environmental display 116 may include one each of a left- side facing and a right-side facing (relative to the front side of primary display 104) wide-angle RGB projector.
- environmental display 116 is located on top of primary display 104, although this is not required. The environmental display may be located at another position proximate to the primary display, or in a position away from the primary display.
- FIG. 1 While the example primary display 104 and environmental display 116 shown in FIG. 1 include 2-D display devices, it will be appreciated that suitable 3-D displays may be used without departing from the scope of the present disclosure.
- user 102 may enjoy an immersive 3-D experience using suitable headgear, such as active shutter glasses (not shown) configured to operate in synchronization with suitable alternate-frame image sequencing at primary display 104 and environmental display 116.
- suitable headgear such as active shutter glasses (not shown) configured to operate in synchronization with suitable alternate-frame image sequencing at primary display 104 and environmental display 116.
- immersive 3-D experiences may be provided with suitable complementary color glasses used to view suitable stereographic images displayed by primary display 104 and environmental display 116.
- user 102 may enjoy an immersive 3-D display experience without using headgear.
- primary display 104 may be equipped with suitable parallax barriers or lenticular lenses to provide an autostereoscopic display while environmental display 116 renders parallax views of the peripheral image in suitably quick succession to accomplish a 3-D display of the peripheral image via "wiggle" stereoscopy.
- any suitable combination of 3-D display techniques including the approaches described above may be employed without departing from the scope of the present disclosure.
- a 3-D primary image may be provided via primary display 104 while a 2-D peripheral image is provided via environmental display 116 or the other way around.
- Interactive computing system 110 is also operatively connected with a depth camera 114.
- depth camera 114 is configured to generate three-dimensional depth information for display environment 100.
- depth camera 114 may be configured as a time-of-flight camera configured to determine spatial distance information by calculating the difference between launch and capture times for emitted and reflected light pulses.
- depth camera 114 may include a three-dimensional scanner configured to collect reflected structured light, such as light patterns emitted by a MEMS laser or infrared light patterns projected by an LCD, LCOS, or DLP projector. It will be understood that, in some embodiments, the light pulses or structured light may be emitted by environmental display 116 or by any suitable light source.
- depth camera 114 may include a plurality of suitable image capture devices to capture three-dimensional depth information within display environment 100.
- depth camera 114 may include each of a forward-facing and a backward-facing (relative to a front-side primary display 104 facing user 102) fisheye image capture device configured to receive reflected light from display environment 100 and provide depth information for a 360-degree field of view surrounding depth camera 114.
- depth camera 114 may include image processing software configured to stitch a panoramic image from a plurality of captured images. In such embodiments, multiple image capture devices may be included in depth camera 114.
- depth camera 114 or a companion camera may also be configured to collect color information from display environment 100, such as by generating color reflectivity information from collected RGB patterns.
- color information may be generated from images collected by a CCD video camera operatively connected with interactive computing system 110 or depth camera 114.
- depth camera 114 shares a common housing with environmental display 116.
- depth camera 114 and environmental display 116 may have a near- common perspective, which may enhance distortion- correction in the peripheral image relative to conditions where depth camera 114 and environmental display 116 are located farther apart.
- depth camera 114 may be a standalone peripheral device opera tively coupled with interactive computing system 110.
- interactive computing system 110 is operatively connected with a user tracking device 118.
- User tracking device 118 may include a suitable depth camera configured to track user movements and features (e.g., head tracking, eye tracking, body tracking, etc.).
- interactive computing system 110 may identify and track a user position for user 102, and act in response to user movements detected by user tracking device 118.
- gestures performed by user 102 while playing a video game running on interactive computing system 110 may be recognized and interpreted as game controls.
- the tracking device 118 allows the user to control the game without the use of conventional, hand-held game controllers.
- user tracking device 118 may track a user's eyes to determine a direction of the user's gaze. For example, a user's eyes may be tracked to comparatively improve the appearance of an image displayed by an autostereoscopic display at primary display 104 or to comparatively enlarge the size of a stereoscopic "sweet spot" of an autostereoscopic display at primary display 104 relative to approaches where a user's eyes are not tracked.
- user tracking device 118 may share a common housing with environmental display 116 and/or depth camera 114.
- depth camera 114 may perform all of the functions of user tracking device 118, or in the alternative, user tracking device 118 may perform all of the functions of depth camera 114.
- one or more of environmental display 116, depth camera 114, and tracking device 118 may be integrated with primary display 104.
- FIG. 2 shows a method 200 of providing a user with an immersive display experience. It will be understood that embodiments of method 200 may be performed using suitable hardware and software such as the hardware and software described herein. Further, it will be appreciated that the order of method 200 is not limiting.
- method 200 comprises displaying the primary image on the primary display, and, at 204, displaying the peripheral image on the environmental display so that the peripheral image appears to be an extension of the primary image.
- the peripheral image may include images of scenery and objects that exhibit the same style and context as scenery and objects depicted in the primary image, so that, within an acceptable tolerance, a user focusing on the primary image perceives the primary image and the peripheral image as forming a whole and complete scene.
- the same virtual object may be partially displayed as part of the primary image and partially displayed as part of the peripheral image.
- FIG. 3 schematically shows an embodiment of a portion of display environment 100 and an embodiment of primary display 104.
- peripheral image 302 is displayed on an environmental surface 112 behind primary display 104 while a primary image 304 is displayed on primary display 104.
- Peripheral image 302 has a lower resolution than primary image 304, schematically illustrated in FIG. 3 by a comparatively larger pixel size for peripheral image 302 than for primary image 304.
- method 200 may comprise, at 206, displaying a distortion- corrected peripheral image.
- the display of the peripheral image may be adjusted to compensate for the topography and/or color of environmental surfaces within the display environment.
- topographical and/or color compensation may be based on a depth map for the display environment used for correcting topographical and geometric distortions in the peripheral image and/or by building a color map for the display environment used for correcting color distortions in the peripheral image.
- method 200 includes, at 208, generating distortion correction from depth, color, and/or perspective information related to the display environment, and, at 210 applying the distortion correction to the peripheral image.
- distortion correction Non-limiting examples of geometric distortion correction, perspective distortion correction, and color distortion corrected are described below.
- applying the distortion correction to the peripheral image 210 may include, at 212, compensating for the topography of an environmental surface so that the peripheral image appears as a geometrically distortion- corrected extension of the primary image.
- geometric distortion correction transformations may be calculated based on depth information and applied to the peripheral image prior to projection to compensate for the topography of environmental surfaces. Such geometric distortion correction transformations may be generated in any suitable way.
- depth information used to generate a geometric distortion correction may be generated by projecting structured light onto environmental surfaces of the display environment and building a depth map from reflected structured light.
- Such depth maps may be generated by a suitable depth camera configured to measure the reflected structured light (or reflected light pulses in scenarios where a time-of-flight depth camera is used to collect depth information).
- structured light may be projected on walls, furniture, and decorative and architectural elements of a user's entertainment room.
- a depth camera may collect structured light reflected by a particular environmental surface to determine the spatial position of the particular environmental surface and/or spatial relationships with other environmental surfaces within the display environment. The spatial positions for several environmental surfaces within the display environment may then be assembled into a depth map for the display environment. While the example above refers to structured light, it will be understood that any suitable light for building a depth map for the display environment may be used. Infrared structured light may be used in some embodiments, while non-visible light pulses configured for use with a time-of-flight depth camera may be used in some other embodiments. Furthermore, time-of-flight depth analysis may be used without departing from the scope of this disclosure.
- the geometric distortion correction may be used by an image correction processor configured to adjust the peripheral image to compensate for the topography of the environmental surface described by the depth information.
- the output of the image correction processor is then output to the environmental display so that the peripheral image appears as a geometrically distortion- corrected extension of the primary image.
- an interactive computing device may multiply the portion of the peripheral image to be displayed on the lamp surface by a suitable correction coefficient.
- pixels for display on the lamp may be adjusted, prior to projection, to form a circularly- shaped region. Once projected on the lamp, the circularly- shaped region would appear as horizontal lines.
- user position information may be used to adjust an apparent perspective of the peripheral image display. Because the depth camera may not be located at the user's location or at the user's eye level, the depth information collected may not represent the depth information perceived by the user. Put another way, the depth camera may not have the same perspective of the display environment as the user has, so that the geometrically corrected peripheral image may still appear slightly incorrect to the user. Thus, in some embodiments, the peripheral image may be further corrected so that the peripheral image appears to be projected from the user position.
- compensating for the topography of the environmental surface at 212 may include compensating for a difference between a perspective of the depth camera at the depth camera position and the user's perspective at the user's position.
- the user's eyes may be tracked by the depth camera or other suitable tracking device to adjust the perspective of the peripheral image.
- the geometric distortion correction transformations described above may include suitable transformations configured to accomplish the 3-D display.
- the geometric distortion correction transformations may include transformations correct for the topography of the environmental surfaces while providing alternating views configured to provide a parallax view of the peripheral image.
- applying the distortion correction to the peripheral image 210 may include, at 214, compensating for the color of an environmental surface so that the peripheral image appears as a color distortion- corrected extension of the primary image.
- color distortion correction transformations may be calculated based on color information and applied to the peripheral image prior to projection to compensate for the color of environmental surfaces. Such color distortion correction transformations may be generated in any suitable way.
- color information used to generate a color distortion correction may be generated by projecting a suitable color pattern onto environmental surfaces of the display environment and building a color map from reflected light.
- Such color maps may be generated by a suitable camera configured to measure color reflectivity.
- an RGB pattern (or any suitable color pattern) may be projected on to the environmental surfaces of the display environment by the environmental display or by any suitable color projection device.
- Light reflected from environmental surfaces of the display environment may be collected (for example, by the depth camera).
- the color information generated from the collected reflected light may be used to build a color map for the display environment.
- the depth camera may perceive that the walls of the user's entertainment room are painted blue. Because an uncorrected projection of blue light displayed on the walls would appear uncolored, the interactive computing device may multiply the portion of the peripheral image to be displayed on the walls by a suitable color correction coefficient. Specifically, pixels for display on the walls may be adjusted, prior to projection, to increase a red content for those pixels. Once projected on the walls, the peripheral image would appear to the user to be blue.
- a color profile of the display environment may be constructed without projecting colored light onto the display environment.
- a camera may be used to capture a color image of the display environment under ambient light, and suitable color corrections may be estimated.
- the color distortion correction transformations described above may include suitable transformations configured to accomplish the 3-D display.
- the color distortion correction transformations may be adjusted to provide a 3-D display to a user wearing glasses having colored lenses, including, but not limited to, amber and blue lenses or red and cyan lenses.
- distortion correction for the peripheral image may be performed at any suitable time and in any suitable order.
- distortion correction may occur at the startup of an immersive display activity and/or at suitable intervals during the immersive display activity.
- distortion correction may be adjusted as the user moves around within the display environment, as light levels change, etc.
- displaying the peripheral image by the environmental display 204 may include, at 216, shielding a portion of the user position from light projected by the environmental display.
- projection of the peripheral image may be actually and/or virtually masked so that a user will perceive relatively less light shining from the peripheral display to the user position. This may protect the user's eyesight and may avoid distracting the user when moving portions of the peripheral image appear to be moving along the user's body.
- an interactive computing device tracks a user position using the depth input received from the depth camera and outputs the peripheral image so that a portion of the user position is shielded from peripheral image light projected from the environmental display.
- shielding a portion of the user position 216 may include determining the user position at 218.
- a user position may be received from a depth camera or other suitable user tracking device.
- receiving the user position may include receiving a user outline.
- user position information may also be used to track a user's head, eyes, etc. when performing the perspective correction described above.
- the user position and/or outline may be identified by the user's motion relative to the environmental surfaces of the display environment, or by any suitable detection method.
- the user position may be tracked over time so that the portion of the peripheral image that is shielded tracks changes in the user position.
- shielding a portion of the user position at 216 may include, at 220, masking a user position from a portion of the peripheral image.
- the portion of the peripheral image that would be displayed at the user position may be identified.
- Such masking may occur by establishing a shielded region of the peripheral image, within which light is not projected.
- pixels in a DLP projection device may be turned off or set to display black in the region of the user's position. It will be understood that corrections for the optical characteristics of the projector and/or for other diffraction conditions may be included when calculating the shielded region.
- the masked region at the projector may have a different appearance from the projected masked region.
- FIGS. 4 and 5 schematically show an embodiment of a display environment 100 in which a peripheral image 302 is being projected at time To (FIG. 4) and at a later time Ti (FIG. 5).
- FIGS. 4 and 5 schematically show an embodiment of a display environment 100 in which a peripheral image 302 is being projected at time To (FIG. 4) and at a later time Ti (FIG. 5).
- the outline of user 102 is shown in both figures, user 102 moving from left to right as time progresses.
- a shielded region 602 tracks the user's head, so that projection light is not directed into the user's eyes. While FIGS. 4 and 5 depict shielded region 602 as a roughly elliptical region, it will be appreciated that shielded region 602 may have any suitable shape and size.
- shielded region 602 may be shaped according to the user's body shape (preventing projection of light onto other portions of the user's body). Further, in some embodiments, shielded region 602 may include a suitable buffer region. Such a buffer region may prevent projected light from leaking onto the user's body within an acceptable tolerance.
- the above described methods and processes may be tied to a computing system including one or more computers.
- the methods and processes described herein may be implemented as a computer application, computer service, computer API, computer library, and/or other computer program product.
- FIG. 6 schematically shows embodiments of primary display 104, depth camera 114, environmental display 116, and user tracking device 118 operatively connected with interactive computing system 110.
- a peripheral input 114a operatively connects depth camera 114 to interactive computing system 110;
- a primary display output 104a operatively connects primary display 104 to interactive computing system 110! and
- an environmental display output 116a operatively connects environmental display 116 to interactive computing system 110.
- one or more of user tracking device 118, primary display 104, environmental display 116, and/or depth camera 114 may be integrated into a multi-functional device.
- one or more of the above described connections may be multi ⁇ functional.
- two or more of the above described connections can be integrated into a common connection.
- suitable connections include USB, USB 2.0, IEEE 1394, HDMI, 802. llx, and/or virtually any other suitable wired or wireless connection.
- Interactive computing system 110 is shown in simplified form. It is to be understood that virtually any computer architecture may be used without departing from the scope of this disclosure.
- interactive computing system 110 may take the form of a mainframe computer, server computer, desktop computer, laptop computer, tablet computer, home entertainment computer, network computing device, mobile computing device, mobile communication device, gaming device, etc.
- Interactive computing system 110 includes a logic subsystem 802 and a data-holding subsystem 804.
- Interactive computing system 110 may also optionally include user input devices such as keyboards, mice, game controllers, cameras, microphones, and/or touch screens, for example.
- Logic subsystem 802 may include one or more physical devices configured to execute one or more instructions.
- the logic subsystem may be configured to execute one or more instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs.
- Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more devices, or otherwise arrive at a desired result.
- the logic subsystem may include one or more processors that are configured to execute software instructions. Additionally or alternatively, the logic subsystem may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. Processors of the logic subsystem may be single core or multicore, and the programs executed thereon may be configured for parallel or distributed processing. The logic subsystem may optionally include individual components that are distributed throughout two or more devices, which may be remotely located and/or configured for coordinated processing. One or more aspects of the logic subsystem may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.
- Data-holding subsystem 804 may include one or more physical, non-transitory, devices configured to hold data and/or instructions executable by the logic subsystem to implement the herein described methods and processes. When such methods and processes are implemented, the state of data-holding subsystem 804 may be transformed (e.g., to hold different data).
- Data-holding subsystem 804 may include removable media and/or built-in devices.
- Data-holding subsystem 804 may include optical memory devices (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory devices (e.g., RAM, EPROM, EEPROM, etc.) and/or magnetic memory devices (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), among others.
- Data-holding subsystem ?? may include devices with one or more of the following characteristics 1 volatile, nonvolatile, dynamic, static, read/write, read-only, random access, sequential access, location addressable, file addressable, and content addressable.
- logic subsystem 802 and data-holding subsystem 804 may be integrated into one or more common devices, such as an application specific integrated circuit or a system on a chip.
- FIG. 6 also shows an aspect of the data-holding subsystem in the form of removable computer-readable storage media 806 which may be used to store and/or transfer data and/or instructions executable to implement the herein described methods and processes.
- Removable computer-readable storage media may take the form of CDs, DVDs, HD-DVDs, Blu-Ray Discs, EEPROMs, and/or floppy disks, among others.
- data-holding subsystem 804 includes one or more physical, non-transitory devices.
- aspects of the instructions described herein may be propagated in a transitory fashion by a pure signal (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for at least a finite duration.
- a pure signal e.g., an electromagnetic signal, an optical signal, etc.
- data and/or other forms of information pertaining to the present disclosure may be propagated by a pure signal.
- the methods described herein may be instantiated via logic subsystem 802 executing instructions held by data-holding subsystem 804. It is to be understood that such methods may take the form of a module, a program and/or an engine. In some embodiments, different modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc.
- module “program,” and “engine” are meant to encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
- Controls And Circuits For Display Device (AREA)
- Processing Or Creating Images (AREA)
- Stored Programmes (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
Abstract
A data-holding subsystem holding instructions executable by a logic subsystem is provided. The instructions are configured to output a primary image to a primary display for display by the primary display, and output a peripheral image to an environmental display for projection by the environmental display on an environmental surface of a display environment so that the peripheral image appears as an extension of the primary image.
Description
IMMERSIVE DISPLAY EXPERIENCE
BACKGROUND
[0001] User enjoyment of video games and related media experiences can be increased by making the gaming experience more realistic. Previous attempts to make the experience more realistic have included switching from two-dimensional to three-dimensional animation techniques, increasing the resolution of game graphics, producing improved sound effects, and creating more natural game controllers.
SUMMARY
[0002] An immersive display environment is provided to a human user by projecting a peripheral image onto environmental surfaces around the user. The peripheral images serve as an extension to a primary image displayed on a primary display.
[0003] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 schematically shows an embodiment of an immersive display environment.
[0005] FIG. 2 shows an example method of providing a user with an immersive display experience.
[0006] FIG. 3 schematically shows an embodiment of a peripheral image displayed as an extension of a primary image.
[0007] FIG. 4 schematically shows an example shielded region of a peripheral image, the shielded region shielding display of the peripheral image at the user position.
[0008] FIG. 5 schematically shows the shielded region of FIG. 4 adjusted to track a movement of the user at a later time.
[0009] FIG. 6 schematically shows an interactive computing system according to an embodiment of the present disclosure.
DETAILED DESCRIPTION
[0010] Interactive media experiences, such as video games, are commonly delivered by a high quality, high resolution display. Such displays are typically the only source of visual content, so that the media experience is bounded by the bezel of the display. Even when focused on the display, the user may perceive architectural and decorative features of the room the display is in via the user's peripheral vision. Such features are typically out of context with respect to the displayed image, muting the entertainment potential of the media experience. Further, because some entertainment experiences engage the user's situational awareness (e.g., in experiences like the video game scenario described above), the ability to perceive motion and identify objects in the peripheral environment (i.e., in a region outside of the high resolution display) may intensify the entertainment experience.
[0011] Various embodiments are described herein that provide the user with an immersive display experience by displaying a primary image on a primary display and a peripheral image that appears, to the user, to be an extension of the primary image.
[0012] FIG. 1 schematically shows an embodiment of a display environment 100. Display environment 100 is depicted as a room configured for leisure and social activities in a user's home. In the example shown in FIG. 1, display environment 100 includes furniture and walls, though it will be understood that various decorative elements and architectural fixtures not shown in FIG. 1 may also be present.
[0013] As shown in Fig. 1, a user 102 is playing a video game using an interactive computing system 110 (such as a gaming console) that outputs a primary image to primary display 104 and projects a peripheral image on environmental surfaces (e.g., walls, furniture, etc.) within display environment 100 via environmental display 116. An embodiment of interactive computing system 110 will be described in more detail below with reference to FIG. 6.
[0014] In the example shown in FIG. 1, a primary image is displayed on primary display 104. As depicted in FIG. 1, primary display 104 is a flat panel display, though it will be appreciated that any suitable display may be used for primary display 104 without departing from the scope of the present disclosure. In the gaming scenario shown in FIG. 1, user 102 is focused on primary images displayed on primary display 104. For example, user 102 may be engaged in attacking video game enemies that are shown on primary display 104.
[0015] As depicted in FIG. 1, interactive computing system 110 is operatively connected with various peripheral devices. For example, interactive computing system 110 is operatively connected with an environmental display 116, which is configured to display a peripheral image on environmental surfaces of the display environment. The peripheral image is configured to appear to be an extension of the primary image displayed on the primary display when viewed by the user. Thus, environmental display 116 may project images that have the same image context as the primary image. As a user perceives the peripheral image with the user's peripheral vision, the user may be situationally aware of images and objects in the peripheral vision while being focused on the primary image.
[0016] In the example shown in FIG. 1, user 102 is focused on the wall displayed on primary display 104 but may be aware of an approaching video game enemy from the user's perception of the peripheral image displayed on environmental surface 112. In some embodiments, the peripheral image is configured so that, to a user, the peripheral image appears to surround the user when projected by the environmental display. Thus, in the context of the gaming scenario shown in FIG. 1, user 102 may turn around and observe an enemy sneaking up from behind.
[0017] In the embodiment shown in FIG. 1, environmental display 116 is a projection display device configured to project a peripheral image in a 360- degree field around environmental display 116. In some embodiments, environmental display 116 may include one each of a left- side facing and a right-side facing (relative to the front side of primary display 104) wide-angle
RGB projector. In FIG. 1, environmental display 116 is located on top of primary display 104, although this is not required. The environmental display may be located at another position proximate to the primary display, or in a position away from the primary display.
[0018] While the example primary display 104 and environmental display 116 shown in FIG. 1 include 2-D display devices, it will be appreciated that suitable 3-D displays may be used without departing from the scope of the present disclosure. For example, in some embodiments, user 102 may enjoy an immersive 3-D experience using suitable headgear, such as active shutter glasses (not shown) configured to operate in synchronization with suitable alternate-frame image sequencing at primary display 104 and environmental display 116. In some embodiments, immersive 3-D experiences may be provided with suitable complementary color glasses used to view suitable stereographic images displayed by primary display 104 and environmental display 116.
[0019] In some embodiments, user 102 may enjoy an immersive 3-D display experience without using headgear. For example, primary display 104 may be equipped with suitable parallax barriers or lenticular lenses to provide an autostereoscopic display while environmental display 116 renders parallax views of the peripheral image in suitably quick succession to accomplish a 3-D display of the peripheral image via "wiggle" stereoscopy. It will be understood that any suitable combination of 3-D display techniques including the approaches described above may be employed without departing from the scope of the present disclosure. Further, it will be appreciated that, in some embodiments, a 3-D primary image may be provided via primary display 104 while a 2-D peripheral image is provided via environmental display 116 or the other way around.
[0020] Interactive computing system 110 is also operatively connected with a depth camera 114. In the embodiment shown in FIG. 1, depth camera 114 is configured to generate three-dimensional depth information for display environment 100. For example, in some embodiments, depth camera 114 may be configured as a time-of-flight camera configured to determine spatial
distance information by calculating the difference between launch and capture times for emitted and reflected light pulses. Alternatively, in some embodiments, depth camera 114 may include a three-dimensional scanner configured to collect reflected structured light, such as light patterns emitted by a MEMS laser or infrared light patterns projected by an LCD, LCOS, or DLP projector. It will be understood that, in some embodiments, the light pulses or structured light may be emitted by environmental display 116 or by any suitable light source.
[0021] In some embodiments, depth camera 114 may include a plurality of suitable image capture devices to capture three-dimensional depth information within display environment 100. For example, in some embodiments, depth camera 114 may include each of a forward-facing and a backward-facing (relative to a front-side primary display 104 facing user 102) fisheye image capture device configured to receive reflected light from display environment 100 and provide depth information for a 360-degree field of view surrounding depth camera 114. Additionally or alternatively, in some embodiments, depth camera 114 may include image processing software configured to stitch a panoramic image from a plurality of captured images. In such embodiments, multiple image capture devices may be included in depth camera 114.
[0022] As explained below, in some embodiments, depth camera 114 or a companion camera (not shown) may also be configured to collect color information from display environment 100, such as by generating color reflectivity information from collected RGB patterns. However, it will be appreciated that other suitable peripheral devices may be used to collect and generate color information without departing from the scope of the present disclosure. For example, in one scenario, color information may be generated from images collected by a CCD video camera operatively connected with interactive computing system 110 or depth camera 114.
[0023] In the embodiment shown in FIG. 1, depth camera 114 shares a common housing with environmental display 116. By sharing a common housing, depth camera 114 and environmental display 116 may have a near-
common perspective, which may enhance distortion- correction in the peripheral image relative to conditions where depth camera 114 and environmental display 116 are located farther apart. However, it will be appreciated that depth camera 114 may be a standalone peripheral device opera tively coupled with interactive computing system 110.
[0024] As shown in the embodiment of FIG. 1, interactive computing system 110 is operatively connected with a user tracking device 118. User tracking device 118 may include a suitable depth camera configured to track user movements and features (e.g., head tracking, eye tracking, body tracking, etc.). In turn, interactive computing system 110 may identify and track a user position for user 102, and act in response to user movements detected by user tracking device 118. Thus, gestures performed by user 102 while playing a video game running on interactive computing system 110 may be recognized and interpreted as game controls. In other words, the tracking device 118 allows the user to control the game without the use of conventional, hand-held game controllers. In some embodiments where a 3-D image is presented to a user, user tracking device 118 may track a user's eyes to determine a direction of the user's gaze. For example, a user's eyes may be tracked to comparatively improve the appearance of an image displayed by an autostereoscopic display at primary display 104 or to comparatively enlarge the size of a stereoscopic "sweet spot" of an autostereoscopic display at primary display 104 relative to approaches where a user's eyes are not tracked.
[0025] It will be appreciated that, in some embodiments, user tracking device 118 may share a common housing with environmental display 116 and/or depth camera 114. In some embodiments, depth camera 114 may perform all of the functions of user tracking device 118, or in the alternative, user tracking device 118 may perform all of the functions of depth camera 114. Furthermore, one or more of environmental display 116, depth camera 114, and tracking device 118 may be integrated with primary display 104.
[0026] FIG. 2 shows a method 200 of providing a user with an immersive display experience. It will be understood that embodiments of method 200 may be performed using suitable hardware and software such as
the hardware and software described herein. Further, it will be appreciated that the order of method 200 is not limiting.
[0027] At 202, method 200 comprises displaying the primary image on the primary display, and, at 204, displaying the peripheral image on the environmental display so that the peripheral image appears to be an extension of the primary image. Put another way, the peripheral image may include images of scenery and objects that exhibit the same style and context as scenery and objects depicted in the primary image, so that, within an acceptable tolerance, a user focusing on the primary image perceives the primary image and the peripheral image as forming a whole and complete scene. In some instances, the same virtual object may be partially displayed as part of the primary image and partially displayed as part of the peripheral image.
[0028] Because a user may be focused and interacting with images displayed on the primary display, in some embodiments, the peripheral image may be displayed at a lower resolution than the primary image without adversely affecting user experience. This may provide an acceptable immersive display environment while reducing computing overhead. For example, FIG. 3 schematically shows an embodiment of a portion of display environment 100 and an embodiment of primary display 104. In the example shown in FIG. 3, peripheral image 302 is displayed on an environmental surface 112 behind primary display 104 while a primary image 304 is displayed on primary display 104. Peripheral image 302 has a lower resolution than primary image 304, schematically illustrated in FIG. 3 by a comparatively larger pixel size for peripheral image 302 than for primary image 304.
[0029] Turning back to FIG. 2, in some embodiments, method 200 may comprise, at 206, displaying a distortion- corrected peripheral image. In such embodiments, the display of the peripheral image may be adjusted to compensate for the topography and/or color of environmental surfaces within the display environment.
[0030] In some of such embodiments, topographical and/or color compensation may be based on a depth map for the display environment used for correcting topographical and geometric distortions in the peripheral image and/or by building a color map for the display environment used for correcting color distortions in the peripheral image. Thus, in such embodiments, method 200 includes, at 208, generating distortion correction from depth, color, and/or perspective information related to the display environment, and, at 210 applying the distortion correction to the peripheral image. Non-limiting examples of geometric distortion correction, perspective distortion correction, and color distortion corrected are described below.
[0031] In some embodiments, applying the distortion correction to the peripheral image 210 may include, at 212, compensating for the topography of an environmental surface so that the peripheral image appears as a geometrically distortion- corrected extension of the primary image. For example, in some embodiments, geometric distortion correction transformations may be calculated based on depth information and applied to the peripheral image prior to projection to compensate for the topography of environmental surfaces. Such geometric distortion correction transformations may be generated in any suitable way.
[0032] In some embodiments, depth information used to generate a geometric distortion correction may be generated by projecting structured light onto environmental surfaces of the display environment and building a depth map from reflected structured light. Such depth maps may be generated by a suitable depth camera configured to measure the reflected structured light (or reflected light pulses in scenarios where a time-of-flight depth camera is used to collect depth information).
[0033] For example, structured light may be projected on walls, furniture, and decorative and architectural elements of a user's entertainment room. A depth camera may collect structured light reflected by a particular environmental surface to determine the spatial position of the particular environmental surface and/or spatial relationships with other environmental surfaces within the display environment. The spatial positions for several
environmental surfaces within the display environment may then be assembled into a depth map for the display environment. While the example above refers to structured light, it will be understood that any suitable light for building a depth map for the display environment may be used. Infrared structured light may be used in some embodiments, while non-visible light pulses configured for use with a time-of-flight depth camera may be used in some other embodiments. Furthermore, time-of-flight depth analysis may be used without departing from the scope of this disclosure.
[0034] Once the geometric distortion correction is generated, it may be used by an image correction processor configured to adjust the peripheral image to compensate for the topography of the environmental surface described by the depth information. The output of the image correction processor is then output to the environmental display so that the peripheral image appears as a geometrically distortion- corrected extension of the primary image.
[0035] For example, because an uncorrected projection of horizontal lines displayed on a cylindrically- shaped lamp included in a display environment would appear as half-circles, an interactive computing device may multiply the portion of the peripheral image to be displayed on the lamp surface by a suitable correction coefficient. Thus, pixels for display on the lamp may be adjusted, prior to projection, to form a circularly- shaped region. Once projected on the lamp, the circularly- shaped region would appear as horizontal lines.
[0036] In some embodiments, user position information may be used to adjust an apparent perspective of the peripheral image display. Because the depth camera may not be located at the user's location or at the user's eye level, the depth information collected may not represent the depth information perceived by the user. Put another way, the depth camera may not have the same perspective of the display environment as the user has, so that the geometrically corrected peripheral image may still appear slightly incorrect to the user. Thus, in some embodiments, the peripheral image may be further corrected so that the peripheral image appears to be projected from the user
position. In such embodiments, compensating for the topography of the environmental surface at 212 may include compensating for a difference between a perspective of the depth camera at the depth camera position and the user's perspective at the user's position. In some embodiments, the user's eyes may be tracked by the depth camera or other suitable tracking device to adjust the perspective of the peripheral image.
[0037] In some embodiments where a 3-D peripheral image is displayed by the environmental display to a user, the geometric distortion correction transformations described above may include suitable transformations configured to accomplish the 3-D display. For example, the geometric distortion correction transformations may include transformations correct for the topography of the environmental surfaces while providing alternating views configured to provide a parallax view of the peripheral image.
[0038] In some embodiments, applying the distortion correction to the peripheral image 210 may include, at 214, compensating for the color of an environmental surface so that the peripheral image appears as a color distortion- corrected extension of the primary image. For example, in some embodiments, color distortion correction transformations may be calculated based on color information and applied to the peripheral image prior to projection to compensate for the color of environmental surfaces. Such color distortion correction transformations may be generated in any suitable way.
[0039] In some embodiments, color information used to generate a color distortion correction may be generated by projecting a suitable color pattern onto environmental surfaces of the display environment and building a color map from reflected light. Such color maps may be generated by a suitable camera configured to measure color reflectivity.
[0040] For example, an RGB pattern (or any suitable color pattern) may be projected on to the environmental surfaces of the display environment by the environmental display or by any suitable color projection device. Light reflected from environmental surfaces of the display environment may be collected (for example, by the depth camera). In some embodiments, the color
information generated from the collected reflected light may be used to build a color map for the display environment.
[0041] For example, based on the reflected RGB pattern, the depth camera may perceive that the walls of the user's entertainment room are painted blue. Because an uncorrected projection of blue light displayed on the walls would appear uncolored, the interactive computing device may multiply the portion of the peripheral image to be displayed on the walls by a suitable color correction coefficient. Specifically, pixels for display on the walls may be adjusted, prior to projection, to increase a red content for those pixels. Once projected on the walls, the peripheral image would appear to the user to be blue.
[0042] In some embodiments, a color profile of the display environment may be constructed without projecting colored light onto the display environment. For example, a camera may be used to capture a color image of the display environment under ambient light, and suitable color corrections may be estimated.
[0043] In some embodiments where a 3-D peripheral image is displayed by the environmental display to a user wearing 3-D headgear, the color distortion correction transformations described above may include suitable transformations configured to accomplish the 3-D display. For example, the color distortion correction transformations may be adjusted to provide a 3-D display to a user wearing glasses having colored lenses, including, but not limited to, amber and blue lenses or red and cyan lenses.
[0044] It will be understood that distortion correction for the peripheral image may be performed at any suitable time and in any suitable order. For example, distortion correction may occur at the startup of an immersive display activity and/or at suitable intervals during the immersive display activity. For example, distortion correction may be adjusted as the user moves around within the display environment, as light levels change, etc.
[0045] In some embodiments, displaying the peripheral image by the environmental display 204 may include, at 216, shielding a portion of the user position from light projected by the environmental display. In other words,
projection of the peripheral image may be actually and/or virtually masked so that a user will perceive relatively less light shining from the peripheral display to the user position. This may protect the user's eyesight and may avoid distracting the user when moving portions of the peripheral image appear to be moving along the user's body.
[0046] In some of such embodiments, an interactive computing device tracks a user position using the depth input received from the depth camera and outputs the peripheral image so that a portion of the user position is shielded from peripheral image light projected from the environmental display. Thus, shielding a portion of the user position 216 may include determining the user position at 218. For example, a user position may be received from a depth camera or other suitable user tracking device. Optionally, in some embodiments, receiving the user position may include receiving a user outline. Further, in some embodiments, user position information may also be used to track a user's head, eyes, etc. when performing the perspective correction described above.
[0047] The user position and/or outline may be identified by the user's motion relative to the environmental surfaces of the display environment, or by any suitable detection method. The user position may be tracked over time so that the portion of the peripheral image that is shielded tracks changes in the user position.
[0048] While the user's position is tracked within the display environment, the peripheral image is adjusted so that the peripheral image is not displayed at the user position. Thus, shielding a portion of the user position at 216 may include, at 220, masking a user position from a portion of the peripheral image. For example, because the user position within the physical space of the display environment is known, and because the depth map described above includes a three-dimensional map of the display environment and of where particular portions of the peripheral image will be displayed within the display environment, the portion of the peripheral image that would be displayed at the user position may be identified.
[0049] Once identified, that portion of the peripheral image may be shielded and/or masked from the peripheral image output. Such masking may occur by establishing a shielded region of the peripheral image, within which light is not projected. For example, pixels in a DLP projection device may be turned off or set to display black in the region of the user's position. It will be understood that corrections for the optical characteristics of the projector and/or for other diffraction conditions may be included when calculating the shielded region. Thus, the masked region at the projector may have a different appearance from the projected masked region.
[0050] FIGS. 4 and 5 schematically show an embodiment of a display environment 100 in which a peripheral image 302 is being projected at time To (FIG. 4) and at a later time Ti (FIG. 5). For illustrative purposes, the outline of user 102 is shown in both figures, user 102 moving from left to right as time progresses. As explained above, a shielded region 602 (shown in outline for illustrative purposes only) tracks the user's head, so that projection light is not directed into the user's eyes. While FIGS. 4 and 5 depict shielded region 602 as a roughly elliptical region, it will be appreciated that shielded region 602 may have any suitable shape and size. For example, shielded region 602 may be shaped according to the user's body shape (preventing projection of light onto other portions of the user's body). Further, in some embodiments, shielded region 602 may include a suitable buffer region. Such a buffer region may prevent projected light from leaking onto the user's body within an acceptable tolerance.
[0051] In some embodiments, the above described methods and processes may be tied to a computing system including one or more computers. In particular, the methods and processes described herein may be implemented as a computer application, computer service, computer API, computer library, and/or other computer program product.
[0052] FIG. 6 schematically shows embodiments of primary display 104, depth camera 114, environmental display 116, and user tracking device 118 operatively connected with interactive computing system 110. In particular, a peripheral input 114a operatively connects depth camera 114 to interactive
computing system 110; a primary display output 104a operatively connects primary display 104 to interactive computing system 110! and an environmental display output 116a operatively connects environmental display 116 to interactive computing system 110. As introduced above, one or more of user tracking device 118, primary display 104, environmental display 116, and/or depth camera 114 may be integrated into a multi-functional device. As such, one or more of the above described connections may be multi¬ functional. In other words, two or more of the above described connections can be integrated into a common connection. Nonlimiting examples of suitable connections include USB, USB 2.0, IEEE 1394, HDMI, 802. llx, and/or virtually any other suitable wired or wireless connection.
[0053] Interactive computing system 110 is shown in simplified form. It is to be understood that virtually any computer architecture may be used without departing from the scope of this disclosure. In different embodiments, interactive computing system 110 may take the form of a mainframe computer, server computer, desktop computer, laptop computer, tablet computer, home entertainment computer, network computing device, mobile computing device, mobile communication device, gaming device, etc.
[0054] Interactive computing system 110 includes a logic subsystem 802 and a data-holding subsystem 804. Interactive computing system 110 may also optionally include user input devices such as keyboards, mice, game controllers, cameras, microphones, and/or touch screens, for example.
[0055] Logic subsystem 802 may include one or more physical devices configured to execute one or more instructions. For example, the logic subsystem may be configured to execute one or more instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more devices, or otherwise arrive at a desired result.
[0056] The logic subsystem may include one or more processors that are configured to execute software instructions. Additionally or alternatively, the logic subsystem may include one or more hardware or firmware logic
machines configured to execute hardware or firmware instructions. Processors of the logic subsystem may be single core or multicore, and the programs executed thereon may be configured for parallel or distributed processing. The logic subsystem may optionally include individual components that are distributed throughout two or more devices, which may be remotely located and/or configured for coordinated processing. One or more aspects of the logic subsystem may be virtualized and executed by remotely accessible networked computing devices configured in a cloud computing configuration.
[0057] Data-holding subsystem 804 may include one or more physical, non-transitory, devices configured to hold data and/or instructions executable by the logic subsystem to implement the herein described methods and processes. When such methods and processes are implemented, the state of data-holding subsystem 804 may be transformed (e.g., to hold different data).
[0058] Data-holding subsystem 804 may include removable media and/or built-in devices. Data-holding subsystem 804 may include optical memory devices (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory devices (e.g., RAM, EPROM, EEPROM, etc.) and/or magnetic memory devices (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), among others. Data-holding subsystem ?? may include devices with one or more of the following characteristics1 volatile, nonvolatile, dynamic, static, read/write, read-only, random access, sequential access, location addressable, file addressable, and content addressable. In some embodiments, logic subsystem 802 and data-holding subsystem 804 may be integrated into one or more common devices, such as an application specific integrated circuit or a system on a chip.
[0059] FIG. 6 also shows an aspect of the data-holding subsystem in the form of removable computer-readable storage media 806 which may be used to store and/or transfer data and/or instructions executable to implement the herein described methods and processes. Removable computer-readable storage media ?? may take the form of CDs, DVDs, HD-DVDs, Blu-Ray Discs, EEPROMs, and/or floppy disks, among others.
[0060] It is to be appreciated that data-holding subsystem 804 includes one or more physical, non-transitory devices. In contrast, in some embodiments aspects of the instructions described herein may be propagated in a transitory fashion by a pure signal (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for at least a finite duration. Furthermore, data and/or other forms of information pertaining to the present disclosure may be propagated by a pure signal.
[0061] In some cases, the methods described herein may be instantiated via logic subsystem 802 executing instructions held by data-holding subsystem 804. It is to be understood that such methods may take the form of a module, a program and/or an engine. In some embodiments, different modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" are meant to encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.
[0062] It is to be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated may be performed in the sequence illustrated, in other sequences, in parallel, or in some cases omitted. Likewise, the order of the above-described processes may be changed.
[0063] The subject matter of the present disclosure includes all novel and nonobvious combinations and subcombinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.
Claims
1. A computer-implemented method for personalizing application processing for a user by an executing application instance comprising:
receiving contextually relevant and personalized content for a user from a context relevant, content aggregation and distribution service wherein the received content is based upon data which the service has received from a different executing application unavailable to the application instance!
receiving a context for the user in the content; and
outputting content which is personalized to the user and relevant to the context of the user.
2. The method of claim 1 further comprising:
the context of the user includes a physical location of the user and one or more people physically present within a vicinity of the user! and
responsive to receiving contextually relevant and personalized content for the one or more people physically present, outputting content on a display device which is personalized for the one or more people in the vicinity of the user.
3. The method of claim 1 further comprising:
the context of the user includes one or more people! and
responsive to receiving contextually relevant and personalized content for the one or more people, the application outputting content on a display device which is personalized for the one or more people and the user.
4. A system for providing personalized content about a user to an application instance for context relevant processing comprising:
one or more datastores storing user profile data including current context data for the user!
one or more servers having access to the one or more datastores and being in communication over communication networks with different communication protocols with computer systems executing online resources! the one or more servers executing software for receiving a request for selected categories of data about the user from an executing application instance!
the one or more servers executing software for searching and gathering content for the selected categories of data about the user from the online resources, wherein the online resources include resources unavailable to the executing application instance! and
the one or more servers executing software for sending the executing application instance content based on the current context data of the user and the content gathered for the selected categories.
5. The system of claim 4 further comprising:
the one or more servers being in communication with one or more client computer devices associated with the user and the servers executing software for determining context data for the user based on context information received from the one or more client computer devices.
6. The system of claim 4 further comprising a database
management system executing on the one or more servers for categorizing content aggregated from the online resources into categories and storing the categories of data in the one or more datastores.
7. The system of claim 4, wherein:
the one or more servers execute software including an application programming interface for receiving the request for selected categories of data about the user! and
the one or more servers execute software including an application programming interface for sending the content based on the current context data of the user.
8. The system of claim 7 wherein the categories of data comprise location data, activity data, availability data, history data, and device data about one or more client devices associated with the user.
9. One or more processor readable storage devices having processor readable code embodied on the one or more processor readable storage devices, the processor readable code for programming one or more processors to perform a method for providing personalized content about a user to an application for context relevant processing, the method comprising:
automatically and continuously gathering content about one or more topics of interest to a user from online resources executing on computer systems accessible via different communication protocols!
receiving a request from an application for data describing a user's interest in one or more topics and a context for the user!
automatically filtering the content gathered for the user based on the application data request from the application, user profile data and a current user context; and
providing contextually relevant content for the user based on the filtering to the requesting application by providing a recommendation based on the filtering for one or more selected categories of data of the application data request wherein the recommendation is based on private preference data of the user from an application different than the requesting application.
10. The one or more processor readable storage devices of claim 9, wherein:
the contextually relevant content includes content based on user data retrieved from another application executing on another user client device associated with the user.
Priority Applications (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020137022983A KR20140014160A (en) | 2011-03-02 | 2012-02-27 | Immersive display experience |
JP2013556783A JP2014509759A (en) | 2011-03-02 | 2012-02-27 | Immersive display experience |
EP12752325.6A EP2681641A4 (en) | 2011-03-02 | 2012-02-27 | Immersive display experience |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/039,179 | 2011-03-02 | ||
US13/039,179 US20120223885A1 (en) | 2011-03-02 | 2011-03-02 | Immersive display experience |
Publications (2)
Publication Number | Publication Date |
---|---|
WO2012118769A2 true WO2012118769A2 (en) | 2012-09-07 |
WO2012118769A9 WO2012118769A9 (en) | 2012-11-22 |
Family
ID=46752990
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2012/026823 WO2012118769A2 (en) | 2011-03-02 | 2012-02-27 | Immersive display experience |
Country Status (8)
Country | Link |
---|---|
US (1) | US20120223885A1 (en) |
EP (1) | EP2681641A4 (en) |
JP (1) | JP2014509759A (en) |
KR (1) | KR20140014160A (en) |
CN (1) | CN102681663A (en) |
AR (1) | AR085517A1 (en) |
TW (1) | TW201244459A (en) |
WO (1) | WO2012118769A2 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2731080A1 (en) * | 2012-11-09 | 2014-05-14 | Sony Computer Entertainment Europe Ltd. | System and method of image rendering |
WO2019002652A2 (en) | 2017-06-27 | 2019-01-03 | Broomx Technologies, S.L. | Method for projecting immersive audiovisual content |
US10523931B2 (en) | 2016-04-07 | 2019-12-31 | Broomx Technologies, S.L. | System for projecting immersive audiovisual content |
US10586306B2 (en) | 2017-03-31 | 2020-03-10 | Korea Advanced Institute Of Science And Technology | Immersive display apparatus and method for creation of peripheral view corresponding to input video |
Families Citing this family (338)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9158116B1 (en) | 2014-04-25 | 2015-10-13 | Osterhout Group, Inc. | Temple and ear horn assembly for headworn computer |
US8427424B2 (en) | 2008-09-30 | 2013-04-23 | Microsoft Corporation | Using physical objects in conjunction with an interactive surface |
US9952664B2 (en) | 2014-01-21 | 2018-04-24 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9366867B2 (en) | 2014-07-08 | 2016-06-14 | Osterhout Group, Inc. | Optical systems for see-through displays |
US9715112B2 (en) | 2014-01-21 | 2017-07-25 | Osterhout Group, Inc. | Suppression of stray light in head worn computing |
US20150277120A1 (en) | 2014-01-21 | 2015-10-01 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US9229233B2 (en) | 2014-02-11 | 2016-01-05 | Osterhout Group, Inc. | Micro Doppler presentations in head worn computing |
US20150205111A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US9400390B2 (en) | 2014-01-24 | 2016-07-26 | Osterhout Group, Inc. | Peripheral lighting for head worn computing |
US9965681B2 (en) | 2008-12-16 | 2018-05-08 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9298007B2 (en) | 2014-01-21 | 2016-03-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9971458B2 (en) | 2009-03-25 | 2018-05-15 | Mep Tech, Inc. | Projection of interactive environment |
US20110256927A1 (en) * | 2009-03-25 | 2011-10-20 | MEP Games Inc. | Projection of interactive game environment |
US20110165923A1 (en) * | 2010-01-04 | 2011-07-07 | Davis Mark L | Electronic circle game system |
US8730309B2 (en) | 2010-02-23 | 2014-05-20 | Microsoft Corporation | Projectors and depth cameras for deviceless augmented reality and interaction |
US9111326B1 (en) | 2010-12-21 | 2015-08-18 | Rawles Llc | Designation of zones of interest within an augmented reality environment |
US9418479B1 (en) | 2010-12-23 | 2016-08-16 | Amazon Technologies, Inc. | Quasi-virtual objects in an augmented reality environment |
US8845110B1 (en) | 2010-12-23 | 2014-09-30 | Rawles Llc | Powered augmented reality projection accessory display device |
US8905551B1 (en) | 2010-12-23 | 2014-12-09 | Rawles Llc | Unpowered augmented reality projection accessory display device |
US9134593B1 (en) | 2010-12-23 | 2015-09-15 | Amazon Technologies, Inc. | Generation and modulation of non-visible structured light for augmented reality projection system |
US8845107B1 (en) | 2010-12-23 | 2014-09-30 | Rawles Llc | Characterization of a scene with structured light |
US9721386B1 (en) | 2010-12-27 | 2017-08-01 | Amazon Technologies, Inc. | Integrated augmented reality environment |
US9607315B1 (en) | 2010-12-30 | 2017-03-28 | Amazon Technologies, Inc. | Complementing operation of display devices in an augmented reality environment |
US9508194B1 (en) | 2010-12-30 | 2016-11-29 | Amazon Technologies, Inc. | Utilizing content output devices in an augmented reality environment |
US9329469B2 (en) | 2011-02-17 | 2016-05-03 | Microsoft Technology Licensing, Llc | Providing an interactive experience using a 3D depth camera and a 3D projector |
US9480907B2 (en) | 2011-03-02 | 2016-11-01 | Microsoft Technology Licensing, Llc | Immersive display with peripheral illusions |
US10972680B2 (en) * | 2011-03-10 | 2021-04-06 | Microsoft Technology Licensing, Llc | Theme-based augmentation of photorepresentative view |
US9007473B1 (en) * | 2011-03-30 | 2015-04-14 | Rawles Llc | Architecture for augmented reality environment |
US9478067B1 (en) | 2011-04-08 | 2016-10-25 | Amazon Technologies, Inc. | Augmented reality environment with secondary sensory feedback |
US9597587B2 (en) | 2011-06-08 | 2017-03-21 | Microsoft Technology Licensing, Llc | Locational node device |
US9996972B1 (en) | 2011-06-10 | 2018-06-12 | Amazon Technologies, Inc. | User/object interactions in an augmented reality environment |
US9921641B1 (en) | 2011-06-10 | 2018-03-20 | Amazon Technologies, Inc. | User/object interactions in an augmented reality environment |
US10008037B1 (en) | 2011-06-10 | 2018-06-26 | Amazon Technologies, Inc. | User/object interactions in an augmented reality environment |
US10595052B1 (en) | 2011-06-14 | 2020-03-17 | Amazon Technologies, Inc. | Dynamic cloud content distribution |
US9723293B1 (en) * | 2011-06-21 | 2017-08-01 | Amazon Technologies, Inc. | Identifying projection surfaces in augmented reality environments |
US9973848B2 (en) * | 2011-06-21 | 2018-05-15 | Amazon Technologies, Inc. | Signal-enhancing beamforming in an augmented reality environment |
US9194938B2 (en) | 2011-06-24 | 2015-11-24 | Amazon Technologies, Inc. | Time difference of arrival determination with direct sound |
US9292089B1 (en) | 2011-08-24 | 2016-03-22 | Amazon Technologies, Inc. | Gestural object selection |
US9462262B1 (en) | 2011-08-29 | 2016-10-04 | Amazon Technologies, Inc. | Augmented reality environment with environmental condition control |
US9380270B1 (en) | 2011-08-31 | 2016-06-28 | Amazon Technologies, Inc. | Skin detection in an augmented reality environment |
US9269152B1 (en) | 2011-09-07 | 2016-02-23 | Amazon Technologies, Inc. | Object detection with distributed sensor array |
US8953889B1 (en) | 2011-09-14 | 2015-02-10 | Rawles Llc | Object datastore in an augmented reality environment |
US9118782B1 (en) | 2011-09-19 | 2015-08-25 | Amazon Technologies, Inc. | Optical interference mitigation |
US9595115B1 (en) | 2011-09-19 | 2017-03-14 | Amazon Technologies, Inc. | Visualizing change in augmented reality environments |
US9349217B1 (en) | 2011-09-23 | 2016-05-24 | Amazon Technologies, Inc. | Integrated community of augmented reality environments |
US9033516B2 (en) * | 2011-09-27 | 2015-05-19 | Qualcomm Incorporated | Determining motion of projection device |
US9628843B2 (en) * | 2011-11-21 | 2017-04-18 | Microsoft Technology Licensing, Llc | Methods for controlling electronic devices using gestures |
US8983089B1 (en) | 2011-11-28 | 2015-03-17 | Rawles Llc | Sound source localization using multiple microphone arrays |
US8887043B1 (en) | 2012-01-17 | 2014-11-11 | Rawles Llc | Providing user feedback in projection environments |
US9418658B1 (en) | 2012-02-08 | 2016-08-16 | Amazon Technologies, Inc. | Configuration of voice controlled assistant |
US9947333B1 (en) | 2012-02-10 | 2018-04-17 | Amazon Technologies, Inc. | Voice interaction architecture with intelligent background noise cancellation |
KR101922589B1 (en) * | 2012-02-15 | 2018-11-27 | 삼성전자주식회사 | Display apparatus and eye tracking method thereof |
KR102038856B1 (en) | 2012-02-23 | 2019-10-31 | 찰스 디. 휴스턴 | System and method for creating an environment and for sharing a location based experience in an environment |
US10600235B2 (en) | 2012-02-23 | 2020-03-24 | Charles D. Huston | System and method for capturing and sharing a location based experience |
US10937239B2 (en) | 2012-02-23 | 2021-03-02 | Charles D. Huston | System and method for creating an environment and for sharing an event |
US9704027B1 (en) | 2012-02-27 | 2017-07-11 | Amazon Technologies, Inc. | Gesture recognition |
US9351089B1 (en) | 2012-03-14 | 2016-05-24 | Amazon Technologies, Inc. | Audio tap detection |
US8662676B1 (en) | 2012-03-14 | 2014-03-04 | Rawles Llc | Automatic projector calibration |
US9338447B1 (en) | 2012-03-14 | 2016-05-10 | Amazon Technologies, Inc. | Calibrating devices by selecting images having a target having fiducial features |
US8898064B1 (en) | 2012-03-19 | 2014-11-25 | Rawles Llc | Identifying candidate passwords from captured audio |
US9111542B1 (en) | 2012-03-26 | 2015-08-18 | Amazon Technologies, Inc. | Audio signal transmission techniques |
US9472005B1 (en) | 2012-04-18 | 2016-10-18 | Amazon Technologies, Inc. | Projection and camera system for augmented reality environment |
US9129375B1 (en) | 2012-04-25 | 2015-09-08 | Rawles Llc | Pose detection |
US8837778B1 (en) | 2012-06-01 | 2014-09-16 | Rawles Llc | Pose tracking |
US9060224B1 (en) | 2012-06-01 | 2015-06-16 | Rawles Llc | Voice controlled assistant with coaxial speaker and microphone arrangement |
US9456187B1 (en) | 2012-06-01 | 2016-09-27 | Amazon Technologies, Inc. | Edge-based pose detection |
US9055237B1 (en) | 2012-06-01 | 2015-06-09 | Rawles Llc | Projection autofocus |
US9800862B2 (en) | 2012-06-12 | 2017-10-24 | The Board Of Trustees Of The University Of Illinois | System and methods for visualizing information |
US9262983B1 (en) | 2012-06-18 | 2016-02-16 | Amazon Technologies, Inc. | Rear projection system with passive display screen |
US9195127B1 (en) | 2012-06-18 | 2015-11-24 | Amazon Technologies, Inc. | Rear projection screen with infrared transparency |
US9892666B1 (en) | 2012-06-20 | 2018-02-13 | Amazon Technologies, Inc. | Three-dimensional model generation |
US9734839B1 (en) * | 2012-06-20 | 2017-08-15 | Amazon Technologies, Inc. | Routing natural language commands to the appropriate applications |
US9330647B1 (en) | 2012-06-21 | 2016-05-03 | Amazon Technologies, Inc. | Digital audio services to augment broadcast radio |
US8885815B1 (en) | 2012-06-25 | 2014-11-11 | Rawles Llc | Null-forming techniques to improve acoustic echo cancellation |
US9280973B1 (en) | 2012-06-25 | 2016-03-08 | Amazon Technologies, Inc. | Navigating content utilizing speech-based user-selectable elements |
US9373338B1 (en) | 2012-06-25 | 2016-06-21 | Amazon Technologies, Inc. | Acoustic echo cancellation processing based on feedback from speech recognizer |
US8971543B1 (en) | 2012-06-25 | 2015-03-03 | Rawles Llc | Voice controlled assistant with stereo sound from two speakers |
US9485556B1 (en) | 2012-06-27 | 2016-11-01 | Amazon Technologies, Inc. | Speaker array for sound imaging |
US9767828B1 (en) | 2012-06-27 | 2017-09-19 | Amazon Technologies, Inc. | Acoustic echo cancellation using visual cues |
US9560446B1 (en) | 2012-06-27 | 2017-01-31 | Amazon Technologies, Inc. | Sound source locator with distributed microphone array |
US10528853B1 (en) | 2012-06-29 | 2020-01-07 | Amazon Technologies, Inc. | Shape-Based Edge Detection |
US9551922B1 (en) | 2012-07-06 | 2017-01-24 | Amazon Technologies, Inc. | Foreground analysis on parametric background surfaces |
US9294746B1 (en) | 2012-07-09 | 2016-03-22 | Amazon Technologies, Inc. | Rotation of a micro-mirror device in a projection and camera system |
US9071771B1 (en) | 2012-07-10 | 2015-06-30 | Rawles Llc | Raster reordering in laser projection systems |
US9317109B2 (en) | 2012-07-12 | 2016-04-19 | Mep Tech, Inc. | Interactive image projection accessory |
US9406170B1 (en) | 2012-07-16 | 2016-08-02 | Amazon Technologies, Inc. | Augmented reality system with activity templates |
US9779757B1 (en) | 2012-07-30 | 2017-10-03 | Amazon Technologies, Inc. | Visual indication of an operational state |
US9786294B1 (en) | 2012-07-30 | 2017-10-10 | Amazon Technologies, Inc. | Visual indication of an operational state |
US8970479B1 (en) | 2012-07-31 | 2015-03-03 | Rawles Llc | Hand gesture detection |
US9052579B1 (en) | 2012-08-01 | 2015-06-09 | Rawles Llc | Remote control of projection and camera system |
US10111002B1 (en) | 2012-08-03 | 2018-10-23 | Amazon Technologies, Inc. | Dynamic audio optimization |
US9641954B1 (en) | 2012-08-03 | 2017-05-02 | Amazon Technologies, Inc. | Phone communication via a voice-controlled device |
US9874977B1 (en) | 2012-08-07 | 2018-01-23 | Amazon Technologies, Inc. | Gesture based virtual devices |
US9704361B1 (en) | 2012-08-14 | 2017-07-11 | Amazon Technologies, Inc. | Projecting content within an environment |
US9779731B1 (en) | 2012-08-20 | 2017-10-03 | Amazon Technologies, Inc. | Echo cancellation based on shared reference signals |
US9329679B1 (en) | 2012-08-23 | 2016-05-03 | Amazon Technologies, Inc. | Projection system with multi-surface projection screen |
US9275302B1 (en) | 2012-08-24 | 2016-03-01 | Amazon Technologies, Inc. | Object detection and identification |
US9548012B1 (en) | 2012-08-29 | 2017-01-17 | Amazon Technologies, Inc. | Adaptive ergonomic keyboard |
US9424840B1 (en) | 2012-08-31 | 2016-08-23 | Amazon Technologies, Inc. | Speech recognition platforms |
US9147399B1 (en) | 2012-08-31 | 2015-09-29 | Amazon Technologies, Inc. | Identification using audio signatures and additional characteristics |
US9726967B1 (en) | 2012-08-31 | 2017-08-08 | Amazon Technologies, Inc. | Display media and extensions to display media |
US9197870B1 (en) | 2012-09-12 | 2015-11-24 | Amazon Technologies, Inc. | Automatic projection focusing |
US9160904B1 (en) | 2012-09-12 | 2015-10-13 | Amazon Technologies, Inc. | Gantry observation feedback controller |
KR101429812B1 (en) * | 2012-09-18 | 2014-08-12 | 한국과학기술원 | Device and method of display extension for television by utilizing external projection apparatus |
US9076450B1 (en) | 2012-09-21 | 2015-07-07 | Amazon Technologies, Inc. | Directed audio for speech recognition |
US9922646B1 (en) | 2012-09-21 | 2018-03-20 | Amazon Technologies, Inc. | Identifying a location of a voice-input device |
US9058813B1 (en) | 2012-09-21 | 2015-06-16 | Rawles Llc | Automated removal of personally identifiable information |
US9805721B1 (en) * | 2012-09-21 | 2017-10-31 | Amazon Technologies, Inc. | Signaling voice-controlled devices |
US9286899B1 (en) | 2012-09-21 | 2016-03-15 | Amazon Technologies, Inc. | User authentication for devices using voice input or audio signatures |
US10175750B1 (en) | 2012-09-21 | 2019-01-08 | Amazon Technologies, Inc. | Projected workspace |
US9355431B1 (en) | 2012-09-21 | 2016-05-31 | Amazon Technologies, Inc. | Image correction for physical projection-surface irregularities |
US9495936B1 (en) | 2012-09-21 | 2016-11-15 | Amazon Technologies, Inc. | Image correction based on projection surface color |
US9127942B1 (en) | 2012-09-21 | 2015-09-08 | Amazon Technologies, Inc. | Surface distance determination using time-of-flight of light |
US8933974B1 (en) | 2012-09-25 | 2015-01-13 | Rawles Llc | Dynamic accommodation of display medium tilt |
US8983383B1 (en) | 2012-09-25 | 2015-03-17 | Rawles Llc | Providing hands-free service to multiple devices |
US9020825B1 (en) | 2012-09-25 | 2015-04-28 | Rawles Llc | Voice gestures |
US9251787B1 (en) | 2012-09-26 | 2016-02-02 | Amazon Technologies, Inc. | Altering audio to improve automatic speech recognition |
US9319816B1 (en) | 2012-09-26 | 2016-04-19 | Amazon Technologies, Inc. | Characterizing environment using ultrasound pilot tones |
US8988662B1 (en) | 2012-10-01 | 2015-03-24 | Rawles Llc | Time-of-flight calculations using a shared light source |
US9762862B1 (en) | 2012-10-01 | 2017-09-12 | Amazon Technologies, Inc. | Optical system with integrated projection and image capture |
US10149077B1 (en) | 2012-10-04 | 2018-12-04 | Amazon Technologies, Inc. | Audio themes |
US9870056B1 (en) | 2012-10-08 | 2018-01-16 | Amazon Technologies, Inc. | Hand and hand pose detection |
US8913037B1 (en) | 2012-10-09 | 2014-12-16 | Rawles Llc | Gesture recognition from depth and distortion analysis |
US9109886B1 (en) | 2012-10-09 | 2015-08-18 | Amazon Technologies, Inc. | Time-of-flight of light calibration |
US9392264B1 (en) * | 2012-10-12 | 2016-07-12 | Amazon Technologies, Inc. | Occluded object recognition |
US9323352B1 (en) | 2012-10-23 | 2016-04-26 | Amazon Technologies, Inc. | Child-appropriate interface selection using hand recognition |
US9978178B1 (en) | 2012-10-25 | 2018-05-22 | Amazon Technologies, Inc. | Hand-based interaction in virtually shared workspaces |
US9281727B1 (en) | 2012-11-01 | 2016-03-08 | Amazon Technologies, Inc. | User device-based control of system functionality |
US9275637B1 (en) | 2012-11-06 | 2016-03-01 | Amazon Technologies, Inc. | Wake word evaluation |
US9685171B1 (en) | 2012-11-20 | 2017-06-20 | Amazon Technologies, Inc. | Multiple-stage adaptive filtering of audio signals |
US9204121B1 (en) | 2012-11-26 | 2015-12-01 | Amazon Technologies, Inc. | Reflector-based depth mapping of a scene |
US9336607B1 (en) | 2012-11-28 | 2016-05-10 | Amazon Technologies, Inc. | Automatic identification of projection surfaces |
US9541125B1 (en) | 2012-11-29 | 2017-01-10 | Amazon Technologies, Inc. | Joint locking mechanism |
US10126820B1 (en) | 2012-11-29 | 2018-11-13 | Amazon Technologies, Inc. | Open and closed hand detection |
US9087520B1 (en) | 2012-12-13 | 2015-07-21 | Rawles Llc | Altering audio based on non-speech commands |
US9271111B2 (en) | 2012-12-14 | 2016-02-23 | Amazon Technologies, Inc. | Response endpoint selection |
US9098467B1 (en) | 2012-12-19 | 2015-08-04 | Rawles Llc | Accepting voice commands based on user identity |
US9047857B1 (en) | 2012-12-19 | 2015-06-02 | Rawles Llc | Voice commands for transitioning between device states |
US9147054B1 (en) | 2012-12-19 | 2015-09-29 | Amazon Technolgies, Inc. | Dialogue-driven user security levels |
US9595997B1 (en) | 2013-01-02 | 2017-03-14 | Amazon Technologies, Inc. | Adaption-based reduction of echo and noise |
US9922639B1 (en) | 2013-01-11 | 2018-03-20 | Amazon Technologies, Inc. | User feedback for speech interactions |
US9466286B1 (en) | 2013-01-16 | 2016-10-11 | Amazong Technologies, Inc. | Transitioning an electronic device between device states |
US9171552B1 (en) | 2013-01-17 | 2015-10-27 | Amazon Technologies, Inc. | Multiple range dynamic level control |
US9159336B1 (en) | 2013-01-21 | 2015-10-13 | Rawles Llc | Cross-domain filtering for audio noise reduction |
US9191742B1 (en) | 2013-01-29 | 2015-11-17 | Rawles Llc | Enhancing audio at a network-accessible computing platform |
US9189850B1 (en) | 2013-01-29 | 2015-11-17 | Amazon Technologies, Inc. | Egomotion estimation of an imaging device |
US8992050B1 (en) | 2013-02-05 | 2015-03-31 | Rawles Llc | Directional projection display |
US9041691B1 (en) | 2013-02-11 | 2015-05-26 | Rawles Llc | Projection surface with reflective elements for non-visible light |
US9201499B1 (en) | 2013-02-11 | 2015-12-01 | Amazon Technologies, Inc. | Object tracking in a 3-dimensional environment |
US9304379B1 (en) | 2013-02-14 | 2016-04-05 | Amazon Technologies, Inc. | Projection display intensity equalization |
US9336602B1 (en) | 2013-02-19 | 2016-05-10 | Amazon Technologies, Inc. | Estimating features of occluded objects |
US9866964B1 (en) | 2013-02-27 | 2018-01-09 | Amazon Technologies, Inc. | Synchronizing audio outputs |
US10289203B1 (en) | 2013-03-04 | 2019-05-14 | Amazon Technologies, Inc. | Detection of an input object on or near a surface |
US9460715B2 (en) | 2013-03-04 | 2016-10-04 | Amazon Technologies, Inc. | Identification using audio signatures and additional characteristics |
US9196067B1 (en) | 2013-03-05 | 2015-11-24 | Amazon Technologies, Inc. | Application specific tracking of projection surfaces |
US9062969B1 (en) | 2013-03-07 | 2015-06-23 | Rawles Llc | Surface distance determination using reflected light |
US9065972B1 (en) | 2013-03-07 | 2015-06-23 | Rawles Llc | User face capture in projection-based systems |
US9465484B1 (en) | 2013-03-11 | 2016-10-11 | Amazon Technologies, Inc. | Forward and backward looking vision system |
US9081418B1 (en) | 2013-03-11 | 2015-07-14 | Rawles Llc | Obtaining input from a virtual user interface |
US10297250B1 (en) | 2013-03-11 | 2019-05-21 | Amazon Technologies, Inc. | Asynchronous transfer of audio data |
US9020144B1 (en) | 2013-03-13 | 2015-04-28 | Rawles Llc | Cross-domain processing for noise and echo suppression |
US10133546B2 (en) | 2013-03-14 | 2018-11-20 | Amazon Technologies, Inc. | Providing content on multiple devices |
US9659577B1 (en) | 2013-03-14 | 2017-05-23 | Amazon Technologies, Inc. | Voice controlled assistant with integrated control knob |
US9721586B1 (en) | 2013-03-14 | 2017-08-01 | Amazon Technologies, Inc. | Voice controlled assistant with light indicator |
US10424292B1 (en) | 2013-03-14 | 2019-09-24 | Amazon Technologies, Inc. | System for recognizing and responding to environmental noises |
US9390500B1 (en) | 2013-03-14 | 2016-07-12 | Amazon Technologies, Inc. | Pointing finger detection |
US9842584B1 (en) | 2013-03-14 | 2017-12-12 | Amazon Technologies, Inc. | Providing content on multiple devices |
US9813808B1 (en) | 2013-03-14 | 2017-11-07 | Amazon Technologies, Inc. | Adaptive directional audio enhancement and selection |
US9429833B1 (en) | 2013-03-15 | 2016-08-30 | Amazon Technologies, Inc. | Projection and camera system with repositionable support structure |
US9101824B2 (en) | 2013-03-15 | 2015-08-11 | Honda Motor Co., Ltd. | Method and system of virtual gaming in a vehicle |
US9689960B1 (en) | 2013-04-04 | 2017-06-27 | Amazon Technologies, Inc. | Beam rejection in multi-beam microphone systems |
US8975854B1 (en) | 2013-04-05 | 2015-03-10 | Rawles Llc | Variable torque control of a stepper motor |
US9781214B2 (en) | 2013-04-08 | 2017-10-03 | Amazon Technologies, Inc. | Load-balanced, persistent connection techniques |
US9304736B1 (en) | 2013-04-18 | 2016-04-05 | Amazon Technologies, Inc. | Voice controlled assistant with non-verbal code entry |
US9491033B1 (en) | 2013-04-22 | 2016-11-08 | Amazon Technologies, Inc. | Automatic content transfer |
EP2797314B1 (en) | 2013-04-25 | 2020-09-23 | Samsung Electronics Co., Ltd | Method and Apparatus for Displaying an Image |
US10514256B1 (en) | 2013-05-06 | 2019-12-24 | Amazon Technologies, Inc. | Single source multi camera vision system |
US9293138B2 (en) | 2013-05-14 | 2016-03-22 | Amazon Technologies, Inc. | Storing state information from network-based user devices |
US9563955B1 (en) | 2013-05-15 | 2017-02-07 | Amazon Technologies, Inc. | Object tracking techniques |
US10002611B1 (en) | 2013-05-15 | 2018-06-19 | Amazon Technologies, Inc. | Asynchronous audio messaging |
US9282403B1 (en) | 2013-05-31 | 2016-03-08 | Amazon Technologies, Inc | User perceived gapless playback |
US9494683B1 (en) | 2013-06-18 | 2016-11-15 | Amazon Technologies, Inc. | Audio-based gesture detection |
US11893603B1 (en) | 2013-06-24 | 2024-02-06 | Amazon Technologies, Inc. | Interactive, personalized advertising |
US9557630B1 (en) | 2013-06-26 | 2017-01-31 | Amazon Technologies, Inc. | Projection system with refractive beam steering |
US9640179B1 (en) | 2013-06-27 | 2017-05-02 | Amazon Technologies, Inc. | Tailoring beamforming techniques to environments |
US9747899B2 (en) | 2013-06-27 | 2017-08-29 | Amazon Technologies, Inc. | Detecting self-generated wake expressions |
US9602922B1 (en) | 2013-06-27 | 2017-03-21 | Amazon Technologies, Inc. | Adaptive echo cancellation |
US9978387B1 (en) | 2013-08-05 | 2018-05-22 | Amazon Technologies, Inc. | Reference signal generation for acoustic echo cancellation |
US9778546B2 (en) | 2013-08-15 | 2017-10-03 | Mep Tech, Inc. | Projector for projecting visible and non-visible images |
US20150067603A1 (en) * | 2013-09-05 | 2015-03-05 | Kabushiki Kaisha Toshiba | Display control device |
US9864576B1 (en) | 2013-09-09 | 2018-01-09 | Amazon Technologies, Inc. | Voice controlled assistant with non-verbal user input |
US9346606B1 (en) | 2013-09-09 | 2016-05-24 | Amazon Technologies, Inc. | Package for revealing an item housed therein |
US9672812B1 (en) | 2013-09-18 | 2017-06-06 | Amazon Technologies, Inc. | Qualifying trigger expressions in speech-based systems |
US9755605B1 (en) | 2013-09-19 | 2017-09-05 | Amazon Technologies, Inc. | Volume control |
US9516081B2 (en) | 2013-09-20 | 2016-12-06 | Amazon Technologies, Inc. | Reduced latency electronic content system |
US9001994B1 (en) | 2013-09-24 | 2015-04-07 | Rawles Llc | Non-uniform adaptive echo cancellation |
US9536493B2 (en) | 2013-09-25 | 2017-01-03 | Samsung Electronics Co., Ltd. | Display apparatus and method of controlling display apparatus |
US9558563B1 (en) | 2013-09-25 | 2017-01-31 | Amazon Technologies, Inc. | Determining time-of-fight measurement parameters |
US10134395B2 (en) | 2013-09-25 | 2018-11-20 | Amazon Technologies, Inc. | In-call virtual assistants |
US9877080B2 (en) | 2013-09-27 | 2018-01-23 | Samsung Electronics Co., Ltd. | Display apparatus and method for controlling thereof |
US9441951B1 (en) | 2013-11-25 | 2016-09-13 | Amazon Technologies, Inc. | Documenting test room configurations |
US9698999B2 (en) | 2013-12-02 | 2017-07-04 | Amazon Technologies, Inc. | Natural language control of secondary device |
US9391575B1 (en) | 2013-12-13 | 2016-07-12 | Amazon Technologies, Inc. | Adaptive loudness control |
US10055190B2 (en) | 2013-12-16 | 2018-08-21 | Amazon Technologies, Inc. | Attribute-based audio channel arbitration |
US9721570B1 (en) | 2013-12-17 | 2017-08-01 | Amazon Technologies, Inc. | Outcome-oriented dialogs on a speech recognition platform |
US10224056B1 (en) | 2013-12-17 | 2019-03-05 | Amazon Technologies, Inc. | Contingent device actions during loss of network connectivity |
US9304674B1 (en) | 2013-12-18 | 2016-04-05 | Amazon Technologies, Inc. | Depth-based display navigation |
US10147441B1 (en) | 2013-12-19 | 2018-12-04 | Amazon Technologies, Inc. | Voice controlled system |
US9304582B1 (en) | 2013-12-19 | 2016-04-05 | Amazon Technologies, Inc. | Object-based color detection and correction |
US9319787B1 (en) | 2013-12-19 | 2016-04-19 | Amazon Technologies, Inc. | Estimation of time delay of arrival for microphone arrays |
US9911414B1 (en) | 2013-12-20 | 2018-03-06 | Amazon Technologies, Inc. | Transient sound event detection |
US9319782B1 (en) | 2013-12-20 | 2016-04-19 | Amazon Technologies, Inc. | Distributed speaker synchronization |
US20160019715A1 (en) | 2014-07-15 | 2016-01-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9299194B2 (en) | 2014-02-14 | 2016-03-29 | Osterhout Group, Inc. | Secure sharing in head worn computing |
US9529195B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US20150228119A1 (en) | 2014-02-11 | 2015-08-13 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US9829707B2 (en) | 2014-08-12 | 2017-11-28 | Osterhout Group, Inc. | Measuring content brightness in head worn computing |
US9746686B2 (en) | 2014-05-19 | 2017-08-29 | Osterhout Group, Inc. | Content position calibration in head worn computing |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US9366868B2 (en) | 2014-09-26 | 2016-06-14 | Osterhout Group, Inc. | See-through computer display systems |
US9671613B2 (en) | 2014-09-26 | 2017-06-06 | Osterhout Group, Inc. | See-through computer display systems |
US11103122B2 (en) | 2014-07-15 | 2021-08-31 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US9841599B2 (en) | 2014-06-05 | 2017-12-12 | Osterhout Group, Inc. | Optical configurations for head-worn see-through displays |
US10191279B2 (en) | 2014-03-17 | 2019-01-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US20150277118A1 (en) | 2014-03-28 | 2015-10-01 | Osterhout Group, Inc. | Sensor dependent content position in head worn computing |
US10649220B2 (en) | 2014-06-09 | 2020-05-12 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9594246B2 (en) | 2014-01-21 | 2017-03-14 | Osterhout Group, Inc. | See-through computer display systems |
US11227294B2 (en) | 2014-04-03 | 2022-01-18 | Mentor Acquisition One, Llc | Sight information collection in head worn computing |
US9575321B2 (en) | 2014-06-09 | 2017-02-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9448409B2 (en) | 2014-11-26 | 2016-09-20 | Osterhout Group, Inc. | See-through computer display systems |
US10684687B2 (en) | 2014-12-03 | 2020-06-16 | Mentor Acquisition One, Llc | See-through computer display systems |
US9753288B2 (en) | 2014-01-21 | 2017-09-05 | Osterhout Group, Inc. | See-through computer display systems |
US9532715B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9494800B2 (en) | 2014-01-21 | 2016-11-15 | Osterhout Group, Inc. | See-through computer display systems |
US11487110B2 (en) | 2014-01-21 | 2022-11-01 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9310610B2 (en) | 2014-01-21 | 2016-04-12 | Osterhout Group, Inc. | See-through computer display systems |
US9836122B2 (en) | 2014-01-21 | 2017-12-05 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
US11892644B2 (en) | 2014-01-21 | 2024-02-06 | Mentor Acquisition One, Llc | See-through computer display systems |
US11737666B2 (en) | 2014-01-21 | 2023-08-29 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9523856B2 (en) | 2014-01-21 | 2016-12-20 | Osterhout Group, Inc. | See-through computer display systems |
US9740280B2 (en) | 2014-01-21 | 2017-08-22 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9766463B2 (en) | 2014-01-21 | 2017-09-19 | Osterhout Group, Inc. | See-through computer display systems |
US9651784B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US11669163B2 (en) | 2014-01-21 | 2023-06-06 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US20150205135A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | See-through computer display systems |
US9811152B2 (en) | 2014-01-21 | 2017-11-07 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9846308B2 (en) | 2014-01-24 | 2017-12-19 | Osterhout Group, Inc. | Haptic systems for head-worn computers |
US9753119B1 (en) | 2014-01-29 | 2017-09-05 | Amazon Technologies, Inc. | Audio and depth based sound source localization |
US9363598B1 (en) | 2014-02-10 | 2016-06-07 | Amazon Technologies, Inc. | Adaptive microphone array compensation |
US9401540B2 (en) | 2014-02-11 | 2016-07-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9852545B2 (en) | 2014-02-11 | 2017-12-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US11132173B1 (en) | 2014-02-20 | 2021-09-28 | Amazon Technologies, Inc. | Network scheduling of stimulus-based actions |
JP6039594B2 (en) * | 2014-02-20 | 2016-12-07 | 株式会社ソニー・インタラクティブエンタテインメント | Information processing apparatus and information processing method |
US9615177B2 (en) | 2014-03-06 | 2017-04-04 | Sphere Optics Company, Llc | Wireless immersive experience capture and viewing |
US9294860B1 (en) | 2014-03-10 | 2016-03-22 | Amazon Technologies, Inc. | Identifying directions of acoustically reflective surfaces |
US9374554B1 (en) | 2014-03-25 | 2016-06-21 | Amazon Technologies, Inc. | Display selection for video conferencing |
US9739609B1 (en) | 2014-03-25 | 2017-08-22 | Amazon Technologies, Inc. | Time-of-flight sensor with configurable phase delay |
US9373318B1 (en) | 2014-03-27 | 2016-06-21 | Amazon Technologies, Inc. | Signal rate synchronization for remote acoustic echo cancellation |
US20160187651A1 (en) | 2014-03-28 | 2016-06-30 | Osterhout Group, Inc. | Safety for a vehicle operator with an hmd |
US9336767B1 (en) | 2014-03-28 | 2016-05-10 | Amazon Technologies, Inc. | Detecting device proximities |
US9607207B1 (en) | 2014-03-31 | 2017-03-28 | Amazon Technologies, Inc. | Plane-fitting edge detection |
US9526115B1 (en) | 2014-04-18 | 2016-12-20 | Amazon Technologies, Inc. | Multiple protocol support in distributed device systems |
US9363616B1 (en) | 2014-04-18 | 2016-06-07 | Amazon Technologies, Inc. | Directional capability testing of audio devices |
US20150309534A1 (en) | 2014-04-25 | 2015-10-29 | Osterhout Group, Inc. | Ear horn assembly for headworn computer |
US9651787B2 (en) | 2014-04-25 | 2017-05-16 | Osterhout Group, Inc. | Speaker assembly for headworn computer |
US9672210B2 (en) | 2014-04-25 | 2017-06-06 | Osterhout Group, Inc. | Language translation with head-worn computing |
US10853589B2 (en) | 2014-04-25 | 2020-12-01 | Mentor Acquisition One, Llc | Language translation with head-worn computing |
US9423842B2 (en) | 2014-09-18 | 2016-08-23 | Osterhout Group, Inc. | Thermal management for head-worn computer |
US20160137312A1 (en) | 2014-05-06 | 2016-05-19 | Osterhout Group, Inc. | Unmanned aerial vehicle launch system |
US10210885B1 (en) | 2014-05-20 | 2019-02-19 | Amazon Technologies, Inc. | Message and user profile indications in speech-based systems |
US10249296B1 (en) | 2014-05-27 | 2019-04-02 | Amazon Technologies, Inc. | Application discovery and selection in language-based systems |
US10663740B2 (en) | 2014-06-09 | 2020-05-26 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10236016B1 (en) | 2014-06-16 | 2019-03-19 | Amazon Technologies, Inc. | Peripheral-based selection of audio sources |
US9185391B1 (en) | 2014-06-17 | 2015-11-10 | Actality, Inc. | Adjustable parallax distance, wide field of view, stereoscopic imaging system |
US9430931B1 (en) | 2014-06-18 | 2016-08-30 | Amazon Technologies, Inc. | Determining user location with remote controller |
US10102195B2 (en) | 2014-06-25 | 2018-10-16 | Amazon Technologies, Inc. | Attribute fill using text extraction |
US9691379B1 (en) | 2014-06-26 | 2017-06-27 | Amazon Technologies, Inc. | Selecting from multiple content sources |
US9368105B1 (en) | 2014-06-26 | 2016-06-14 | Amazon Technologies, Inc. | Preventing false wake word detections with a voice-controlled device |
US9548066B2 (en) | 2014-08-11 | 2017-01-17 | Amazon Technologies, Inc. | Voice application architecture |
US10325591B1 (en) * | 2014-09-05 | 2019-06-18 | Amazon Technologies, Inc. | Identifying and suppressing interfering audio content |
US9456276B1 (en) | 2014-09-30 | 2016-09-27 | Amazon Technologies, Inc. | Parameter selection for audio beamforming |
CN104501001B (en) * | 2014-11-28 | 2016-11-23 | 广景科技有限公司 | A kind of intelligence projection bulb and interactive and intelligence projecting method thereof |
US9684172B2 (en) | 2014-12-03 | 2017-06-20 | Osterhout Group, Inc. | Head worn computer display systems |
USD743963S1 (en) | 2014-12-22 | 2015-11-24 | Osterhout Group, Inc. | Air mouse |
USD751552S1 (en) | 2014-12-31 | 2016-03-15 | Osterhout Group, Inc. | Computer glasses |
USD753114S1 (en) | 2015-01-05 | 2016-04-05 | Osterhout Group, Inc. | Air mouse |
US9898078B2 (en) * | 2015-01-12 | 2018-02-20 | Dell Products, L.P. | Immersive environment correction display and method |
EP3251343A4 (en) * | 2015-01-30 | 2018-09-05 | Ent. Services Development Corporation LP | Room capture and projection |
WO2016122582A1 (en) | 2015-01-30 | 2016-08-04 | Hewlett Packard Enterprise Development Lp | Relationship preserving projection of digital objects |
US10878775B2 (en) | 2015-02-17 | 2020-12-29 | Mentor Acquisition One, Llc | See-through computer display systems |
US20160239985A1 (en) | 2015-02-17 | 2016-08-18 | Osterhout Group, Inc. | See-through computer display systems |
WO2016157996A1 (en) | 2015-03-31 | 2016-10-06 | ソニー株式会社 | Information processing device, information processing method, program, and image display system |
US10642349B2 (en) * | 2015-05-21 | 2020-05-05 | Sony Interactive Entertainment Inc. | Information processing apparatus |
US10139966B2 (en) | 2015-07-22 | 2018-11-27 | Osterhout Group, Inc. | External user interface for head worn computing |
NZ742883A (en) * | 2015-10-26 | 2023-04-28 | Liang Kong | Immersive all-in-one pc system |
US10850116B2 (en) | 2016-12-30 | 2020-12-01 | Mentor Acquisition One, Llc | Head-worn therapy device |
US10591728B2 (en) | 2016-03-02 | 2020-03-17 | Mentor Acquisition One, Llc | Optical systems for head-worn computers |
US10667981B2 (en) | 2016-02-29 | 2020-06-02 | Mentor Acquisition One, Llc | Reading assistance system for visually impaired |
US9826299B1 (en) | 2016-08-22 | 2017-11-21 | Osterhout Group, Inc. | Speaker systems for head-worn computer systems |
US9880441B1 (en) | 2016-09-08 | 2018-01-30 | Osterhout Group, Inc. | Electrochromic systems for head-worn computer systems |
US10824253B2 (en) | 2016-05-09 | 2020-11-03 | Mentor Acquisition One, Llc | User interface systems for head-worn computers |
US10466491B2 (en) | 2016-06-01 | 2019-11-05 | Mentor Acquisition One, Llc | Modular systems for head-worn computers |
US10684478B2 (en) | 2016-05-09 | 2020-06-16 | Mentor Acquisition One, Llc | User interface systems for head-worn computers |
US9910284B1 (en) | 2016-09-08 | 2018-03-06 | Osterhout Group, Inc. | Optical systems for head-worn computers |
US10134198B2 (en) | 2016-04-19 | 2018-11-20 | Adobe Systems Incorporated | Image compensation for an occluding direct-view augmented reality system |
AU2016412141B2 (en) * | 2016-06-14 | 2022-03-03 | Razer (Asia-Pacific) Pte. Ltd. | Image processing devices, methods for controlling an image processing device, and computer-readable media |
US10690936B2 (en) | 2016-08-29 | 2020-06-23 | Mentor Acquisition One, Llc | Adjustable nose bridge assembly for headworn computer |
US10438264B1 (en) | 2016-08-31 | 2019-10-08 | Amazon Technologies, Inc. | Artificial intelligence feature extraction service for products |
US20180077437A1 (en) | 2016-09-09 | 2018-03-15 | Barrie Hansen | Parallel Video Streaming |
US10135950B2 (en) | 2016-10-10 | 2018-11-20 | Google Llc | Creating a cinematic storytelling experience using network-addressable devices |
US10819952B2 (en) * | 2016-10-11 | 2020-10-27 | Sony Interactive Entertainment LLC | Virtual reality telepresence |
USD840395S1 (en) | 2016-10-17 | 2019-02-12 | Osterhout Group, Inc. | Head-worn computer |
US10004984B2 (en) * | 2016-10-31 | 2018-06-26 | Disney Enterprises, Inc. | Interactive in-room show and game system |
EP3494458B1 (en) * | 2016-12-14 | 2021-12-01 | Samsung Electronics Co., Ltd. | Display apparatus and method for controlling the display apparatus |
USD864959S1 (en) | 2017-01-04 | 2019-10-29 | Mentor Acquisition One, Llc | Computer glasses |
US10780358B1 (en) * | 2017-03-22 | 2020-09-22 | Intuitive Research And Technology Corporation | Virtual reality arena system |
US10908679B2 (en) * | 2017-04-24 | 2021-02-02 | Intel Corporation | Viewing angles influenced by head and body movements |
US10719947B2 (en) * | 2017-05-19 | 2020-07-21 | Faro Technologies, Inc. | Three-dimensional measurement device with annotation features |
EP3422707A1 (en) * | 2017-06-29 | 2019-01-02 | Vestel Elektronik Sanayi ve Ticaret A.S. | Display system and method |
US10578869B2 (en) | 2017-07-24 | 2020-03-03 | Mentor Acquisition One, Llc | See-through computer display systems with adjustable zoom cameras |
US11409105B2 (en) | 2017-07-24 | 2022-08-09 | Mentor Acquisition One, Llc | See-through computer display systems |
US10422995B2 (en) | 2017-07-24 | 2019-09-24 | Mentor Acquisition One, Llc | See-through computer display systems with stray light management |
US10969584B2 (en) | 2017-08-04 | 2021-04-06 | Mentor Acquisition One, Llc | Image expansion optic for head-worn computer |
TWI642973B (en) * | 2017-09-12 | 2018-12-01 | 晶將數位多媒體科技股份有限公司 | 3D floating stereoscopic image creation and display device |
US10515637B1 (en) | 2017-09-19 | 2019-12-24 | Amazon Technologies, Inc. | Dynamic speech processing |
US10080051B1 (en) * | 2017-10-25 | 2018-09-18 | TCL Research America Inc. | Method and system for immersive information presentation |
US11194464B1 (en) | 2017-11-30 | 2021-12-07 | Amazon Technologies, Inc. | Display control using objects |
US10713007B2 (en) | 2017-12-12 | 2020-07-14 | Amazon Technologies, Inc. | Architecture for a hub configured to control a second device while a connection to a remote system is unavailable |
US10859831B1 (en) * | 2018-05-16 | 2020-12-08 | Facebook Technologies, Llc | Systems and methods for safely operating a mobile virtual reality system |
US10997963B1 (en) * | 2018-05-17 | 2021-05-04 | Amazon Technologies, Inc. | Voice based interaction based on context-based directives |
US20200014909A1 (en) | 2018-07-03 | 2020-01-09 | Faro Technologies, Inc. | Handheld three dimensional scanner with autofocus or autoaperture |
US10540797B1 (en) | 2018-08-02 | 2020-01-21 | Disney Enterprises, Inc. | Image customization using a persona |
GB2593833B (en) * | 2018-11-06 | 2022-09-14 | Lucasfilm Entertainment Co Ltd | Immersive content production system |
JP2020098273A (en) * | 2018-12-18 | 2020-06-25 | ソニーセミコンダクタソリューションズ株式会社 | Image display device |
TWI747333B (en) * | 2020-06-17 | 2021-11-21 | 光時代科技有限公司 | Interaction method based on optical communictation device, electric apparatus, and computer readable storage medium |
WO2022220707A1 (en) * | 2021-04-12 | 2022-10-20 | Хальдун Саид Аль-Зубейди | Virtual teleport room |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070126864A1 (en) | 2005-12-05 | 2007-06-07 | Kiran Bhat | Synthesizing three-dimensional surround visual field |
WO2007134456A1 (en) | 2006-05-24 | 2007-11-29 | Smart Technologies Ulc | Method and apparatus for inhibiting a subject's eyes from being exposed to projected light |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6720949B1 (en) * | 1997-08-22 | 2004-04-13 | Timothy R. Pryor | Man machine interfaces and applications |
JP3880561B2 (en) * | 2002-09-05 | 2007-02-14 | 株式会社ソニー・コンピュータエンタテインメント | Display system |
CA2464569A1 (en) * | 2003-04-16 | 2004-10-16 | Universite De Montreal | Single or multi-projector for arbitrary surfaces without calibration nor reconstruction |
JP4594308B2 (en) * | 2003-08-19 | 2010-12-08 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Visual content signal display device and method for displaying visual content signal |
US7077529B2 (en) * | 2004-02-20 | 2006-07-18 | L-3 Communications Corporation | Masked image projection system and method |
WO2005082075A2 (en) * | 2004-02-25 | 2005-09-09 | The University Of North Carolina At Chapel Hill | Systems and methods for imperceptibly embedding structured light patterns in projected color images |
EP1784978B1 (en) * | 2004-08-30 | 2010-11-10 | Bauhaus-Universität Weimar | Method and device for representing a digital image on a surface which is non-trivial in terms of its geometry and photometry |
US8130330B2 (en) * | 2005-12-05 | 2012-03-06 | Seiko Epson Corporation | Immersive surround visual fields |
JP2007264633A (en) * | 2006-03-28 | 2007-10-11 | Seiko Epson Corp | Surround visual field system, method for synthesizing surround visual field relating to input stream, and surround visual field controller |
MX2008012473A (en) * | 2006-03-31 | 2008-10-10 | Koninkl Philips Electronics Nv | Adaptive rendering of video content based on additional frames of content. |
US7972005B2 (en) * | 2007-04-02 | 2011-07-05 | Agere Systems Inc. | Computer projector method and apparatus having a safety feature for blacking out a portion of the image being projected onto a person |
JP2009031334A (en) * | 2007-07-24 | 2009-02-12 | Sharp Corp | Projector and projection method for projector |
US8488129B2 (en) * | 2007-10-05 | 2013-07-16 | Artec Group, Inc. | Combined object capturing system and display device and associated method |
US20090128783A1 (en) * | 2007-11-15 | 2009-05-21 | Yueh-Hong Shih | Ocular-protection projector device |
JP5431312B2 (en) * | 2008-05-21 | 2014-03-05 | パナソニック株式会社 | projector |
-
2011
- 2011-03-02 US US13/039,179 patent/US20120223885A1/en not_active Abandoned
-
2012
- 2012-02-27 WO PCT/US2012/026823 patent/WO2012118769A2/en unknown
- 2012-02-27 EP EP12752325.6A patent/EP2681641A4/en not_active Withdrawn
- 2012-02-27 KR KR1020137022983A patent/KR20140014160A/en not_active Application Discontinuation
- 2012-02-27 JP JP2013556783A patent/JP2014509759A/en active Pending
- 2012-02-29 AR ARP120100660A patent/AR085517A1/en not_active Application Discontinuation
- 2012-03-01 CN CN2012100517451A patent/CN102681663A/en active Pending
- 2012-03-02 TW TW101107044A patent/TW201244459A/en unknown
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070126864A1 (en) | 2005-12-05 | 2007-06-07 | Kiran Bhat | Synthesizing three-dimensional surround visual field |
WO2007134456A1 (en) | 2006-05-24 | 2007-11-29 | Smart Technologies Ulc | Method and apparatus for inhibiting a subject's eyes from being exposed to projected light |
Non-Patent Citations (1)
Title |
---|
See also references of EP2681641A4 |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2731080A1 (en) * | 2012-11-09 | 2014-05-14 | Sony Computer Entertainment Europe Ltd. | System and method of image rendering |
US10523931B2 (en) | 2016-04-07 | 2019-12-31 | Broomx Technologies, S.L. | System for projecting immersive audiovisual content |
US10586306B2 (en) | 2017-03-31 | 2020-03-10 | Korea Advanced Institute Of Science And Technology | Immersive display apparatus and method for creation of peripheral view corresponding to input video |
WO2019002652A2 (en) | 2017-06-27 | 2019-01-03 | Broomx Technologies, S.L. | Method for projecting immersive audiovisual content |
WO2019002652A3 (en) * | 2017-06-27 | 2020-05-22 | Broomx Technologies, S.L. | Method for projecting immersive audiovisual content |
Also Published As
Publication number | Publication date |
---|---|
US20120223885A1 (en) | 2012-09-06 |
WO2012118769A9 (en) | 2012-11-22 |
JP2014509759A (en) | 2014-04-21 |
EP2681641A4 (en) | 2014-08-27 |
EP2681641A2 (en) | 2014-01-08 |
KR20140014160A (en) | 2014-02-05 |
TW201244459A (en) | 2012-11-01 |
CN102681663A (en) | 2012-09-19 |
AR085517A1 (en) | 2013-10-09 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2681641A2 (en) | Immersive display experience | |
US10497175B2 (en) | Augmented reality virtual monitor | |
US10502959B2 (en) | Realtime lens aberration correction from eye tracking | |
US10372209B2 (en) | Eye tracking enabling 3D viewing | |
US9734633B2 (en) | Virtual environment generating system | |
US20150312561A1 (en) | Virtual 3d monitor | |
KR101925658B1 (en) | Volumetric video presentation | |
JP5967343B2 (en) | Display system and method for optimizing display based on active tracking | |
US9480907B2 (en) | Immersive display with peripheral illusions | |
US20130141419A1 (en) | Augmented reality with realistic occlusion | |
JP4903888B2 (en) | Image display device, image display method, and image correction method | |
US20120218253A1 (en) | Adjusting 3d effects for wearable viewing devices | |
US20120038635A1 (en) | 3-d rendering for a rotated viewer | |
EP3308539A1 (en) | Display for stereoscopic augmented reality | |
WO2012021129A1 (en) | 3d rendering for a rotated viewer | |
JP2012059008A (en) | Program, information storage medium and image generation system | |
JP5222407B2 (en) | Image display device, image display method, and image correction method | |
US20220232201A1 (en) | Image generation system and method | |
US9609313B2 (en) | Enhanced 3D display method and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 12752325 Country of ref document: EP Kind code of ref document: A2 |
|
ENP | Entry into the national phase |
Ref document number: 20137022983 Country of ref document: KR Kind code of ref document: A |
|
ENP | Entry into the national phase |
Ref document number: 2013556783 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |