US20180157322A1 - Display apparatus and method using portable electronic device - Google Patents
Display apparatus and method using portable electronic device Download PDFInfo
- Publication number
- US20180157322A1 US20180157322A1 US15/623,052 US201715623052A US2018157322A1 US 20180157322 A1 US20180157322 A1 US 20180157322A1 US 201715623052 A US201715623052 A US 201715623052A US 2018157322 A1 US2018157322 A1 US 2018157322A1
- Authority
- US
- United States
- Prior art keywords
- image
- focus
- display
- display apparatus
- electronic device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
- G02B7/02—Mountings, adjusting means, or light-tight connections, for optical elements for lenses
- G02B7/04—Mountings, adjusting means, or light-tight connections, for optical elements for lenses with mechanism for focusing or varying magnification
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
- G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
- G09G5/39—Control of the bit-mapped memory
- G09G5/391—Resolution modifying circuits, e.g. variable screen formats
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0147—Head-up displays characterised by optical features comprising a device modifying the resolution of the displayed image
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0149—Head-up displays characterised by mechanical features
- G02B2027/0154—Head-up displays characterised by mechanical features with movable elements
- G02B2027/0159—Head-up displays characterised by mechanical features with movable elements with mechanical means other than scaning means for positioning the whole image
Definitions
- the present disclosure relates generally to virtual reality; and more specifically, to display apparatuses using portable electronic devices. Furthermore, the present disclosure also relates to methods of displaying, via the aforementioned display apparatuses.
- the simulated environment enhances the user's experience of reality around him/her by providing the user with a feeling of immersion in the simulated environment using contemporary techniques such as stereoscopy.
- the user may use a device, such as a virtual reality device, for experiencing such simulated environment.
- virtual reality devices include near field displays. Examples of such virtual reality devices include, head mounted virtual reality devices, virtual reality glasses, and so forth.
- the virtual reality devices include binocular virtual reality devices having one display per eye of the user. In such an instance, both displays of a binocular virtual reality device display different two-dimensional images (also known as stereograms) to the eyes of the user for creating an illusion of depth by combining the different two-dimensional images.
- conventional virtual reality devices have certain limitations. Firstly, the virtual reality devices are required to include high resolution displays to allow for provision of the feeling of immersion and better awareness of the simulated environment to the user. Consequently, the virtual reality devices comprising such sophisticated displays are expensive. Secondly, processors employed in conventional virtual reality devices often lack substantially advanced processing functionality to minimize processing latency therein. Consequently, a lag in processing reduces the user's experience of the simulated environment.
- the present disclosure seeks to provide a display apparatus, comprising at least one focus display, and a processor coupled to the at least one focus display, wherein the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled to the portable electronic device.
- the present disclosure also seeks to provide a method of displaying, via such a display apparatus.
- the present disclosure seeks to provide a solution to the existing problem of expensive displays and significant processing latency within virtual reality devices.
- An aim of the present disclosure is to provide a solution that overcomes at least partially the problems encountered in the prior art, and provides a robust, efficient, and cost-effective display apparatus for the virtual reality devices.
- an embodiment of the present disclosure provides a display apparatus comprising:
- the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled with the portable electronic device, and wherein a processor of the portable electronic device is configured to render a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees,
- the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
- an embodiment of the present disclosure provides a method of displaying, via a display apparatus comprising at least one focus display, the display apparatus being detachably attached to a portable electronic device and being communicably coupled with the portable electronic device, the method comprising:
- Embodiments of the present disclosure substantially eliminate or at least partially address the aforementioned problems in the prior art, and enables minimization of processing latency within virtual reality devices, thereby, enhancing a user's experience of simulated environments presented by such virtual reality devices.
- FIG. 1 is a schematic illustration of a display system comprising a display apparatus and a portable electronic device, in accordance with an embodiment of the present disclosure
- FIGS. 2 and 3 are block diagrams of architectures of a display apparatus and a portable electronic device, in accordance with different embodiments of the present disclosure
- FIG. 4 is an exemplary implementation of a display system, in accordance with an embodiment of the present disclosure.
- FIG. 5 illustrates steps of a method of displaying via the display apparatus, in accordance with an embodiment of the present disclosure.
- an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent.
- a non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.
- an embodiment of the present disclosure provides a display apparatus comprising:
- the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled with the portable electronic device, and wherein a processor of the portable electronic device is configured to render a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees,
- the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
- an embodiment of the present disclosure provides a method of displaying, via a display apparatus comprising at least one focus display, the display apparatus being detachably attached to a portable electronic device and being communicably coupled with the portable electronic device, the method comprising:
- the present disclosure provides the aforementioned display apparatus and the aforementioned method of displaying via the display apparatus.
- the display apparatus described herein is robust, light-weight and cost-effective. Furthermore, since processing functionality (namely, processing burden) is shared between the processors of the display apparatus and the portable electronic device, processing latency in rendering a simulated environment is reduced significantly. Consequently, such minimization of the processing latency enhances a user's experience of the simulated environment.
- the term “portable electronic device” relates to portable electronic equipment that comprises hardware, software, firmware, or a combination of these, operable to implement at least a processing functionality and a rendering functionality. It will be appreciated that the processor of the portable electronic device is operable to implement the aforesaid processing functionality and the display of the portable electronic device is operable to implement the aforesaid rendering functionality. Examples of the portable electronic device include, but are not limited to, a smartphone, a phablet, a personal digital assistant (PDA).
- PDA personal digital assistant
- the display of the portable electronic device is selected from the group consisting of: a Liquid Crystal Display (LCD), a Light Emitting Diode (LED)-based display, an Organic LED (OLED)-based display, a micro OLED-based display, a Liquid Crystal on Silicon (LCoS)-based display, and a Digital Light Processing® (DLP)-based display.
- LCD Liquid Crystal Display
- LED Light Emitting Diode
- OLED Organic LED
- micro OLED-based display a micro OLED-based display
- LCDoS Liquid Crystal on Silicon
- DLP Digital Light Processing®
- the portable electronic device may also be referred to as a “hand-held device”, “mobile device”, and the like.
- the display apparatus is arranged to be detachably attached and to be communicably coupled to the portable electronic device.
- the portable electronic device is mounted on (namely, attached to) the display apparatus.
- the display apparatus supports the mounted portable electronic device thereon.
- the portable electronic device is slid into a cavity on a side of the display apparatus.
- the cavity on the side of the display apparatus is shaped so as to accommodate the portable electronic device.
- the portable electronic device is attached in a manner that the display of the portable electronic device is oriented towards (namely, faces) eyes of the user, when the display apparatus is worn by the user.
- a display system is operable to act as a device, for example, such as a virtual reality device, for presenting the simulated environment to the user, when the display apparatus is worn by the user.
- the term “angular width” refers to an angular width of a given projection as seen from the user's eyes, when the display apparatus is worn by the user.
- the angular width of the projection of the rendered context image is greater than 220 degrees.
- the angular width of a projection of the rendered context image is, for example, such as from 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160 or 170 degrees up to 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210 or 220 degrees.
- the angular width of a projection of the rendered focus image is, for example, such as from 5, 10, 15, 20, 25, 30, 35, 40, 45 or 50 degrees up to 15, 20, 25, 30, 35, 40, 45, 50, 55 or 60 degrees.
- the context image relates to a wide image of the visual scene to be rendered and projected via the display of the portable electronic device.
- the focus image relates to another image depicting a part (namely, a portion) of the visual scene, to be rendered and projected via the display apparatus.
- the focus image is dimensionally smaller than the context image.
- the context and focus images collectively constitute the visual scene upon combination of projections thereof.
- the term ‘focus display’ used herein relates to a display (namely, screen) adapted to facilitate rendering of the focus image thereon.
- the at least one focus display is adapted to receive a projection of the focus image thereon.
- the at least one focus display is selected from the group consisting of: a Liquid Crystal Display (LCD), a Light Emitting Diode (LED)-based display, an Organic LED (OLED)-based display, a micro OLED-based display, a Liquid Crystal on Silicon (LCoS)-based display, and a Digital Light Processing® (DLP)-based display.
- LCD Liquid Crystal Display
- LED Light Emitting Diode
- OLED Organic LED
- micro OLED Organic LED
- LCD Liquid Crystal on Silicon
- DLP Digital Light Processing®
- the at least one focus display is implemented by way of at least one projector and at least one projection screen.
- one focus display may be implemented by way of one projector and one projection screen associated with the one projector.
- the processor of the display apparatus is hardware, software, firmware or a combination of these, suitable for controlling operation of the display apparatus.
- the processor of the display apparatus controls operation of the display apparatus to process and display (namely, project) the focus image onto the eyes, specifically, onto fovea of the eyes of the user.
- the processor of the portable electronic device is hardware, software, firmware or a combination of these, suitable for controlling operation of the portable electronic device.
- the processor of the portable electronic device controls operation of the portable electronic device to process and display (namely, project) the context image onto the eyes, specifically, onto retina of the eyes of the user.
- the processing functionality (namely, processing power) of the processor of the portable electronic device is higher in comparison with processing functionality (namely, processing power) of the processor of the display apparatus.
- the processor of the display apparatus is configured to receive the focus image from the processor of the portable electronic device.
- the processor of the portable electronic device is configured to communicate the focus image in a suitable format including, but not limited to, Moving Pictures Experts Group (MPEG), Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), Portable Network Graphics (PNG), Graphics Interchange Format (GIF), and Bitmap file format (BMP).
- MPEG Moving Pictures Experts Group
- JPEG Joint Photographic Experts Group
- TIFF Tagged Image File Format
- PNG Portable Network Graphics
- GIF Graphics Interchange Format
- BMP Bitmap file format
- the display apparatus further comprises means for detecting a gaze direction, wherein the processor of the display apparatus is configured to communicate the detected gaze direction to the processor of the portable electronic device,
- processor of the portable electronic device is configured to:
- the focus image substantially corresponds to the region of visual accuracy of the input image
- the processor of the portable electronic device communicates the focus image to the processor of the display apparatus in a compressed format to avoid lag in processing. Subsequently, in such an instance, the processor of the display apparatus may decompress the compressed focus image prior to rendering thereof on the at least one focus display of the display apparatus.
- the processor of the portable electronic device partially processes the focus image prior to communication thereof to the processor of the display apparatus.
- the portable electronic device comprises an image sensor for capturing the input image.
- the processor of the portable communication devices performs image processing functions (for example, such as low pass filtering, colour processing, and gamma correction) on the input image.
- the means for detecting a gaze direction relates to specialized equipment for measuring a direction of gaze of the eye and movement of the eye.
- the means for detecting the gaze direction may also be referred to as “eye tracker”.
- Examples of the means for detecting the gaze direction include, but are not limited to, contact lenses with sensors, and cameras monitoring position of pupil of the eye.
- the region of visual accuracy is a region of interest (namely, a fixation point) within the input image, and is projected onto fovea of the eye. Furthermore, the region of visual accuracy is region of focus within the input image. It will be appreciated that the region of visual accuracy relates to a region resolved to a much greater detail as compared to other regions of the input image, when the input image is viewed by a human visual system.
- the context image generated by the processor of the portable electronic device, corresponds to a low-resolution representation of the input image.
- the context image includes the region of visual accuracy of the input image along with remaining region of the input image. Specifically, size of the context image is larger than size of the focus image since the focus image corresponds to only a portion of the context image whereat the detected gaze direction of the eye is focused.
- the masking of the region of the context image is performed by the processor of the portable electronic device to hide (namely, obscure) the region of the context image corresponding to the region of visual accuracy of the input image.
- the processor of the portable electronic device may hide (namely, obscure) the region of the context image corresponding to the region of visual accuracy of the input image.
- pixels of the context image corresponding to the region of visual accuracy of the input image may be dimmed for masking.
- the display apparatus further comprises means for detecting a gaze direction, wherein the processor of the display apparatus is configured to:
- the focus image substantially corresponds to the region of visual accuracy of the input image
- the processor of the portable electronic device is configured to receive the context image from the processor of the display apparatus.
- the display apparatus further comprises at least one optical combiner for combining the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image.
- the combined projections of the rendered context and focus images collectively constitute a projection of the input image. It will be appreciated that the context image and focus image are rendered substantially simultaneously in order to avoid time lag during combination of projections thereof.
- the at least one optical combiner comprises at least one first optical element and at least one first actuator for adjusting a location of the projection of the rendered focus image on the at least one first optical element, wherein the processor of the display apparatus is configured to control the at least one first actuator to move at least one of:
- the at least one focus display with respect to the at least one first optical element of the at least one optical combiner
- the at least one first actuator moves the at least one focus display when the gaze direction of the eye shifts from one direction to another.
- the at least one first actuator may move the at least one focus display closer or away from the at least one first optical element.
- the at least one first actuator may move the at least one focus display laterally with respect to the at least one first optical element.
- the at least one first actuator may tilt and/or rotate the at least one focus display with respect to the at least one first optical element.
- the processor of the display apparatus controls the at least one first actuator by generating an actuation signal (for example, such as electric current, hydraulic pressure, and so forth).
- the at least one second optical element changes the optical path of the projection of the rendered focus image on the at least one first optical element in order to facilitate projection of the rendered focus image on and around the fovea of the eye even on occurrence of shift in the gaze direction.
- the at least one first actuator moves the at least one first optical element to combine optical paths of the projections of the rendered context and focus images. It will be appreciated that such arrangement of the at least one first optical element facilitates projection of the rendered focus image on and around the fovea of the eye, and projection of the rendered context image is projected on a retina of the eye, of which the fovea is just a small part.
- the at least one first optical element of the at least one optical combiner is implemented by way of at least one of: a semi-transparent mirror, a semi-transparent film, a prism, a polarizer, an optical waveguide.
- the display apparatus further comprises at least one focusing lens that is positioned on an optical path between the at least one first optical element and the at least one focus display, and at least one second actuator for moving the at least one focusing lens with respect to the at least one focus display.
- the processor of the display apparatus is configured to control the at least one second actuator to adjust a focus of the projection of the rendered focus image.
- the at least one focusing lens utilizes properties thereof to adjust the focus of the projection of the rendered focus image by changing the optical path thereof.
- the at least one second actuator is controlled by the processor of the display apparatus to adjust the focus of the projection of rendered focus image on the at least one first optical element.
- the processor of the portable electronic device is configured to control the at least one second actuator to adjust a focus of the projection of the rendered focus image.
- the present description also relates to the method as described above.
- the various embodiments and variants disclosed above apply mutatis mutandis to the method.
- FIG. 1 illustrated is a schematic illustration of a display system 100 comprising a display apparatus 102 and a portable electronic device 104 , in accordance with an embodiment of the present disclosure.
- the display apparatus 102 is arranged to be detachably attached to a portable electronic device 104 and to be communicably coupled with the portable electronic device 104 .
- the portable electronic device 104 is arranged in a cavity 106 of the display apparatus 102 .
- the display system 100 is operable to act as a device, for example, such as a virtual reality device, for presenting a simulated environment to a user, when the display apparatus 102 is worn by the user.
- FIG. 1 includes simplified arrangement for implementation of the display system 100 for sake of clarity, which should not unduly limit the scope of the claims herein.
- the person skilled in the art will recognize many variations, alternatives, and modifications of embodiments of the present disclosure.
- the display apparatus 202 comprises at least one focus display 206 , and a processor 208 coupled to the at least one focus display 206 .
- the display apparatus 202 is arranged to be detachably attached to a portable electronic device 204 and to be communicably coupled with the portable electronic device 104 .
- the portable electronic device 204 comprises a processor 210 configured to render a context image at a display 212 of the portable electronic device 204 .
- the display apparatus 302 comprises at least one focus display 306 , and a processor 308 coupled to the at least one focus display 306 .
- the display apparatus 302 further comprises means for detecting a gaze direction 310 .
- the display apparatus 302 is arranged to be detachably attached to a portable electronic device 304 and to be communicably coupled with the portable electronic device 304 .
- the portable electronic device 304 comprises a processor 312 configured to render a context image at a display 314 of the portable electronic device 304 .
- FIG. 4 illustrated is an exemplary implementation of a display system (such as the display system 100 , shown in FIG. 1 ), in accordance with an embodiment of the present disclosure.
- the display system is shown to include a display 402 of a portable electronic device (such as the portable electronic device 104 of FIG. 1 ) to render a context image, at least one focus display 404 of a display apparatus (such as the display apparatus 102 of FIG. 1 ) to render a focus image, and at least one optical combiner 406 .
- at least one focusing lens 408 is positioned on an optical path of projections of the context and focus images.
- the at least one focusing lens 408 is one of an enlarging or a shrinking (namely, reducing) lens.
- a focus image is rendered at, at least one focus display of the display apparatus, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees.
- a context image is rendered at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees.
- the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
- the method 500 further comprises receiving, at the display apparatus, the focus image from the portable electronic device.
- the display apparatus further comprises means for detecting a gaze direction
- the method 500 further comprises detecting, at the display apparatus, a gaze direction, communicating the detected gaze direction to the portable electronic device, obtaining an input image at the portable electronic device, and using the detected gaze direction to determine a region of visual accuracy of the input image, processing the input image to generate the context image and the focus image at the portable electronic device, the context image having a first resolution and the focus image having a second resolution, wherein a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked, the focus image substantially corresponds to the region of visual accuracy of the input image, and the second resolution is higher than the first resolution, and communicating the focus image from the portable electronic device to the display apparatus.
- the display apparatus further comprises means for detecting a gaze direction
- the method 500 further comprises detecting a gaze direction, obtaining an input image at the display apparatus, and using the detected gaze direction to determine a region of visual accuracy of the input image, processing the input image to generate the context image and the focus image at the display apparatus, the context image having a first resolution and the focus image having a second resolution, wherein a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked, the focus image substantially corresponds to the region of visual accuracy of the input image, and the second resolution is higher than the first resolution, and communicating the context image from the display apparatus to the portable electronic device.
- the method 500 further comprises using at least one optical combiner of the display apparatus to combine the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image.
- the at least one optical combiner comprises at least one first optical element
- the method 500 further comprises adjusting a location of the projection of the rendered focus image on the at least one first optical element, the adjusting being performed by controlling at least one first actuator of the at least one optical combiner to move at least one of the at least one focus display with respect to the at least one first optical element of the at least one optical combiner, at least one second optical element of the at least one optical combiner with respect to the at least one first optical element, the at least one second optical element being positioned on an optical path between the at least one first optical element and the at least one focus display, the at least one first optical element of the at least one optical combiner.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Human Computer Interaction (AREA)
- Transforming Electric Information Into Light Information (AREA)
- Controls And Circuits For Display Device (AREA)
- Control Of Indicators Other Than Cathode Ray Tubes (AREA)
Abstract
A display apparatus and a method of displaying, via the display apparatus. The display apparatus includes at least one focus display, and a processor coupled to the at least one focus display. The processor of the display apparatus is configured to render a focus image at the at least one focus display. The display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled with the portable electronic device. A processor of the portable electronic device is configured to render a context image at a display of the portable electronic device. The projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
Description
- This application is a continuation-in-part of U.S. patent application Ser. No. 15/366424, titled “DISPLAY APPARATUS AND METHOD OF DISPLAYING USING FOCUS AND CONTEXT DISPLAYS” and filed on Dec. 1, 2016, which is incorporated herein by reference.
- The present disclosure relates generally to virtual reality; and more specifically, to display apparatuses using portable electronic devices. Furthermore, the present disclosure also relates to methods of displaying, via the aforementioned display apparatuses.
- In recent times, there has been rapid increase in use of technologies such as virtual reality, augmented reality, and so forth, for presenting a simulated environment (or a virtual world) to a user. Specifically, the simulated environment enhances the user's experience of reality around him/her by providing the user with a feeling of immersion in the simulated environment using contemporary techniques such as stereoscopy.
- Typically, the user may use a device, such as a virtual reality device, for experiencing such simulated environment. Optionally, such virtual reality devices include near field displays. Examples of such virtual reality devices include, head mounted virtual reality devices, virtual reality glasses, and so forth. Furthermore, typically, the virtual reality devices include binocular virtual reality devices having one display per eye of the user. In such an instance, both displays of a binocular virtual reality device display different two-dimensional images (also known as stereograms) to the eyes of the user for creating an illusion of depth by combining the different two-dimensional images. However, conventional virtual reality devices have certain limitations. Firstly, the virtual reality devices are required to include high resolution displays to allow for provision of the feeling of immersion and better awareness of the simulated environment to the user. Consequently, the virtual reality devices comprising such sophisticated displays are expensive. Secondly, processors employed in conventional virtual reality devices often lack substantially advanced processing functionality to minimize processing latency therein. Consequently, a lag in processing reduces the user's experience of the simulated environment.
- Therefore, in light of the foregoing discussion, there exists a need to overcome the aforementioned drawbacks associated with conventional virtual reality devices.
- The present disclosure seeks to provide a display apparatus, comprising at least one focus display, and a processor coupled to the at least one focus display, wherein the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled to the portable electronic device. The present disclosure also seeks to provide a method of displaying, via such a display apparatus. The present disclosure seeks to provide a solution to the existing problem of expensive displays and significant processing latency within virtual reality devices. An aim of the present disclosure is to provide a solution that overcomes at least partially the problems encountered in the prior art, and provides a robust, efficient, and cost-effective display apparatus for the virtual reality devices.
- In one aspect, an embodiment of the present disclosure provides a display apparatus comprising:
-
- at least one focus display; and
- a processor coupled to the at least one focus display, wherein the processor is configured to render a focus image at the at least one focus display, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees,
- wherein the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled with the portable electronic device, and wherein a processor of the portable electronic device is configured to render a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees,
- further wherein the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
- In another aspect, an embodiment of the present disclosure provides a method of displaying, via a display apparatus comprising at least one focus display, the display apparatus being detachably attached to a portable electronic device and being communicably coupled with the portable electronic device, the method comprising:
-
- rendering a focus image at the at least one focus display of the display apparatus, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees;
- rendering a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees; and
- combining the projection of the rendered context image with the projection of the rendered focus image to create a visual scene.
- Embodiments of the present disclosure substantially eliminate or at least partially address the aforementioned problems in the prior art, and enables minimization of processing latency within virtual reality devices, thereby, enhancing a user's experience of simulated environments presented by such virtual reality devices.
- Additional aspects, advantages, features and objects of the present disclosure would be made apparent from the drawings and the detailed description of the illustrative embodiments construed in conjunction with the appended claims that follow.
- It will be appreciated that features of the present disclosure are susceptible to being combined in various combinations without departing from the scope of the present disclosure as defined by the appended claims.
- The summary above, as well as the following detailed description of illustrative embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of illustrating the present disclosure, exemplary constructions of the disclosure are shown in the drawings. However, the present disclosure is not limited to specific methods and instrumentalities disclosed herein. Moreover, those in the art will understand that the drawings are not to scale. Wherever possible, like elements have been indicated by identical numbers.
- Embodiments of the present disclosure will now be described, by way of example only, with reference to the following diagrams wherein:
-
FIG. 1 is a schematic illustration of a display system comprising a display apparatus and a portable electronic device, in accordance with an embodiment of the present disclosure; -
FIGS. 2 and 3 are block diagrams of architectures of a display apparatus and a portable electronic device, in accordance with different embodiments of the present disclosure; -
FIG. 4 is an exemplary implementation of a display system, in accordance with an embodiment of the present disclosure; and -
FIG. 5 illustrates steps of a method of displaying via the display apparatus, in accordance with an embodiment of the present disclosure. - In the accompanying drawings, an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent. A non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.
- The following detailed description illustrates embodiments of the present disclosure and ways in which they can be implemented. Although some modes of carrying out the present disclosure have been disclosed, those skilled in the art would recognize that other embodiments for carrying out or practicing the present disclosure are also possible.
- In one aspect, an embodiment of the present disclosure provides a display apparatus comprising:
-
- at least one focus display; and
- a processor coupled to the at least one focus display, wherein the processor is configured to render a focus image at the at least one focus display, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees,
- wherein the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled with the portable electronic device, and wherein a processor of the portable electronic device is configured to render a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees,
- further wherein the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
- In another aspect, an embodiment of the present disclosure provides a method of displaying, via a display apparatus comprising at least one focus display, the display apparatus being detachably attached to a portable electronic device and being communicably coupled with the portable electronic device, the method comprising:
-
- rendering a focus image at the at least one focus display of the display apparatus, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees;
- rendering a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees; and
- combining the projection of the rendered context image with the projection of the rendered focus image to create a visual scene.
- The present disclosure provides the aforementioned display apparatus and the aforementioned method of displaying via the display apparatus. The display apparatus described herein is robust, light-weight and cost-effective. Furthermore, since processing functionality (namely, processing burden) is shared between the processors of the display apparatus and the portable electronic device, processing latency in rendering a simulated environment is reduced significantly. Consequently, such minimization of the processing latency enhances a user's experience of the simulated environment.
- Throughout the present disclosure, the term “portable electronic device” relates to portable electronic equipment that comprises hardware, software, firmware, or a combination of these, operable to implement at least a processing functionality and a rendering functionality. It will be appreciated that the processor of the portable electronic device is operable to implement the aforesaid processing functionality and the display of the portable electronic device is operable to implement the aforesaid rendering functionality. Examples of the portable electronic device include, but are not limited to, a smartphone, a phablet, a personal digital assistant (PDA).
- Optionally, the display of the portable electronic device is selected from the group consisting of: a Liquid Crystal Display (LCD), a Light Emitting Diode (LED)-based display, an Organic LED (OLED)-based display, a micro OLED-based display, a Liquid Crystal on Silicon (LCoS)-based display, and a Digital Light Processing® (DLP)-based display.
- It is to be understood that the portable electronic device may also be referred to as a “hand-held device”, “mobile device”, and the like.
- As mentioned previously, the display apparatus is arranged to be detachably attached and to be communicably coupled to the portable electronic device. In an implementation, the portable electronic device is mounted on (namely, attached to) the display apparatus. Optionally, in such an instance, the display apparatus supports the mounted portable electronic device thereon. In another implementation, the portable electronic device is slid into a cavity on a side of the display apparatus. Optionally, the cavity on the side of the display apparatus is shaped so as to accommodate the portable electronic device. Furthermore, optionally, the portable electronic device is attached in a manner that the display of the portable electronic device is oriented towards (namely, faces) eyes of the user, when the display apparatus is worn by the user.
- It will be appreciated that the aforesaid arrangement of the display apparatus and the portable electronic device collectively constitutes a display system. Furthermore, such a display system is operable to act as a device, for example, such as a virtual reality device, for presenting the simulated environment to the user, when the display apparatus is worn by the user.
- Throughout the present disclosure, the term “angular width” refers to an angular width of a given projection as seen from the user's eyes, when the display apparatus is worn by the user. Optionally, the angular width of the projection of the rendered context image is greater than 220 degrees. Furthermore, optionally, the angular width of a projection of the rendered context image is, for example, such as from 40, 50, 60, 70, 80, 90, 100, 110, 120, 130, 140, 150, 160 or 170 degrees up to 70, 80, 90, 100, 110, 120, 130, 140, 150, 160, 170, 180, 190, 200, 210 or 220 degrees. Moreover, optionally, the angular width of a projection of the rendered focus image is, for example, such as from 5, 10, 15, 20, 25, 30, 35, 40, 45 or 50 degrees up to 15, 20, 25, 30, 35, 40, 45, 50, 55 or 60 degrees.
- It is to be understood that the context image relates to a wide image of the visual scene to be rendered and projected via the display of the portable electronic device. Furthermore, the focus image relates to another image depicting a part (namely, a portion) of the visual scene, to be rendered and projected via the display apparatus. Moreover, the focus image is dimensionally smaller than the context image. Furthermore, the context and focus images collectively constitute the visual scene upon combination of projections thereof.
- Throughout the present disclosure, the term ‘focus display’ used herein relates to a display (namely, screen) adapted to facilitate rendering of the focus image thereon. Moreover, optionally, the at least one focus display is adapted to receive a projection of the focus image thereon. Optionally, the at least one focus display is selected from the group consisting of: a Liquid Crystal Display (LCD), a Light Emitting Diode (LED)-based display, an Organic LED (OLED)-based display, a micro OLED-based display, a Liquid Crystal on Silicon (LCoS)-based display, and a Digital Light Processing® (DLP)-based display.
- Optionally, the at least one focus display is implemented by way of at least one projector and at least one projection screen. For example, one focus display may be implemented by way of one projector and one projection screen associated with the one projector.
- Throughout the present disclosure, the processor of the display apparatus is hardware, software, firmware or a combination of these, suitable for controlling operation of the display apparatus. In such an instance, the processor of the display apparatus controls operation of the display apparatus to process and display (namely, project) the focus image onto the eyes, specifically, onto fovea of the eyes of the user.
- Similarly, throughout the present disclosure, the processor of the portable electronic device is hardware, software, firmware or a combination of these, suitable for controlling operation of the portable electronic device. In such an instance, the processor of the portable electronic device controls operation of the portable electronic device to process and display (namely, project) the context image onto the eyes, specifically, onto retina of the eyes of the user.
- Optionally, the processing functionality (namely, processing power) of the processor of the portable electronic device is higher in comparison with processing functionality (namely, processing power) of the processor of the display apparatus.
- Optionally, the processor of the display apparatus is configured to receive the focus image from the processor of the portable electronic device. Specifically, the processor of the portable electronic device is configured to communicate the focus image in a suitable format including, but not limited to, Moving Pictures Experts Group (MPEG), Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), Portable Network Graphics (PNG), Graphics Interchange Format (GIF), and Bitmap file format (BMP).
- In an embodiment, the display apparatus further comprises means for detecting a gaze direction, wherein the processor of the display apparatus is configured to communicate the detected gaze direction to the processor of the portable electronic device,
- wherein the processor of the portable electronic device is configured to:
- (a) obtain an input image, and use the detected gaze direction to determine a region of visual accuracy of the input image;
- (b) process the input image to generate the context image and the focus image, the context image having a first resolution and the focus image having a second resolution, wherein:
- a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked,
- (ii) the focus image substantially corresponds to the region of visual accuracy of the input image, and
- (iii) the second resolution is higher than the first resolution; and
- (c) communicate the focus image to the processor of the display apparatus. In such an instance, the aforesaid processing functionality is implemented by the processor of the portable electronic device, beneficially, reducing processing burden on the processor of the display apparatus.
- Optionally, the processor of the portable electronic device communicates the focus image to the processor of the display apparatus in a compressed format to avoid lag in processing. Subsequently, in such an instance, the processor of the display apparatus may decompress the compressed focus image prior to rendering thereof on the at least one focus display of the display apparatus.
- Optionally, in this regard, the processor of the portable electronic device partially processes the focus image prior to communication thereof to the processor of the display apparatus. Optionally, the portable electronic device comprises an image sensor for capturing the input image. Optionally, in this regard, the processor of the portable communication devices performs image processing functions (for example, such as low pass filtering, colour processing, and gamma correction) on the input image.
- Throughout the present disclosure, the means for detecting a gaze direction relates to specialized equipment for measuring a direction of gaze of the eye and movement of the eye. The means for detecting the gaze direction may also be referred to as “eye tracker”. Examples of the means for detecting the gaze direction include, but are not limited to, contact lenses with sensors, and cameras monitoring position of pupil of the eye.
- Optionally, the region of visual accuracy is a region of interest (namely, a fixation point) within the input image, and is projected onto fovea of the eye. Furthermore, the region of visual accuracy is region of focus within the input image. It will be appreciated that the region of visual accuracy relates to a region resolved to a much greater detail as compared to other regions of the input image, when the input image is viewed by a human visual system.
- Optionally, the context image, generated by the processor of the portable electronic device, corresponds to a low-resolution representation of the input image. Moreover, optionally, the context image includes the region of visual accuracy of the input image along with remaining region of the input image. Specifically, size of the context image is larger than size of the focus image since the focus image corresponds to only a portion of the context image whereat the detected gaze direction of the eye is focused.
- Optionally, the masking of the region of the context image is performed by the processor of the portable electronic device to hide (namely, obscure) the region of the context image corresponding to the region of visual accuracy of the input image. For example, pixels of the context image corresponding to the region of visual accuracy of the input image may be dimmed for masking.
- In another embodiment, the display apparatus further comprises means for detecting a gaze direction, wherein the processor of the display apparatus is configured to:
- (a) obtain an input image, and use the detected gaze direction to determine a region of visual accuracy of the input image;
- (b) process the input image to generate the context image and the focus image, the context image having a first resolution and the focus image having a second resolution, wherein:
- a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked,
- (ii) the focus image substantially corresponds to the region of visual accuracy of the input image, and
- (iii) the second resolution is higher than the first resolution; and
- (c) communicate the context image to the processor of the portable electronic device.
- Optionally, the processor of the portable electronic device is configured to receive the context image from the processor of the display apparatus.
- Optionally, the display apparatus further comprises at least one optical combiner for combining the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image. Moreover, optionally, the combined projections of the rendered context and focus images collectively constitute a projection of the input image. It will be appreciated that the context image and focus image are rendered substantially simultaneously in order to avoid time lag during combination of projections thereof.
- Optionally, the at least one optical combiner comprises at least one first optical element and at least one first actuator for adjusting a location of the projection of the rendered focus image on the at least one first optical element, wherein the processor of the display apparatus is configured to control the at least one first actuator to move at least one of:
- the at least one focus display with respect to the at least one first optical element of the at least one optical combiner,
- (ii) at least one second optical element of the at least one optical combiner with respect to the at least one first optical element, the at least one second optical element being positioned on an optical path between the at least one first optical element and the at least one focus display,
- (iii) the at least one first optical element of the at least one optical combiner.
- Optionally, the at least one first actuator moves the at least one focus display when the gaze direction of the eye shifts from one direction to another. In an example, the at least one first actuator may move the at least one focus display closer or away from the at least one first optical element. In another example, the at least one first actuator may move the at least one focus display laterally with respect to the at least one first optical element. In yet another example, the at least one first actuator may tilt and/or rotate the at least one focus display with respect to the at least one first optical element. Moreover, optionally, the processor of the display apparatus controls the at least one first actuator by generating an actuation signal (for example, such as electric current, hydraulic pressure, and so forth).
- Optionally, the at least one second optical element changes the optical path of the projection of the rendered focus image on the at least one first optical element in order to facilitate projection of the rendered focus image on and around the fovea of the eye even on occurrence of shift in the gaze direction.
- Optionally, the at least one first actuator moves the at least one first optical element to combine optical paths of the projections of the rendered context and focus images. It will be appreciated that such arrangement of the at least one first optical element facilitates projection of the rendered focus image on and around the fovea of the eye, and projection of the rendered context image is projected on a retina of the eye, of which the fovea is just a small part. In an embodiment, the at least one first optical element of the at least one optical combiner is implemented by way of at least one of: a semi-transparent mirror, a semi-transparent film, a prism, a polarizer, an optical waveguide.
- Optionally, the display apparatus further comprises at least one focusing lens that is positioned on an optical path between the at least one first optical element and the at least one focus display, and at least one second actuator for moving the at least one focusing lens with respect to the at least one focus display. In such an embodiment, the processor of the display apparatus is configured to control the at least one second actuator to adjust a focus of the projection of the rendered focus image. Moreover, optionally, the at least one focusing lens utilizes properties thereof to adjust the focus of the projection of the rendered focus image by changing the optical path thereof. Furthermore, optionally, the at least one second actuator is controlled by the processor of the display apparatus to adjust the focus of the projection of rendered focus image on the at least one first optical element. In another embodiment, the processor of the portable electronic device is configured to control the at least one second actuator to adjust a focus of the projection of the rendered focus image.
- The present description also relates to the method as described above. The various embodiments and variants disclosed above apply mutatis mutandis to the method.
- Referring to
FIG. 1 , illustrated is a schematic illustration of adisplay system 100 comprising a display apparatus 102 and a portableelectronic device 104, in accordance with an embodiment of the present disclosure. The display apparatus 102 is arranged to be detachably attached to a portableelectronic device 104 and to be communicably coupled with the portableelectronic device 104. Specifically, the portableelectronic device 104 is arranged in acavity 106 of the display apparatus 102. Furthermore, thedisplay system 100 is operable to act as a device, for example, such as a virtual reality device, for presenting a simulated environment to a user, when the display apparatus 102 is worn by the user. - It may be understood by a person skilled in the art that the
FIG. 1 includes simplified arrangement for implementation of thedisplay system 100 for sake of clarity, which should not unduly limit the scope of the claims herein. The person skilled in the art will recognize many variations, alternatives, and modifications of embodiments of the present disclosure. - Referring to
FIG. 2 , illustrated is a block diagram of architecture of a display apparatus 202 and a portableelectronic device 204, in accordance with an embodiment of the present disclosure. As shown, the display apparatus 202 comprises at least onefocus display 206, and aprocessor 208 coupled to the at least onefocus display 206. The display apparatus 202 is arranged to be detachably attached to a portableelectronic device 204 and to be communicably coupled with the portableelectronic device 104. Furthermore, the portableelectronic device 204 comprises aprocessor 210 configured to render a context image at adisplay 212 of the portableelectronic device 204. - Referring to
FIG. 3 , illustrated is a block diagram of architecture of adisplay apparatus 302 and a portable electronic device 304, in accordance with another embodiment of the present disclosure. Thedisplay apparatus 302 comprises at least one focus display 306, and a processor 308 coupled to the at least one focus display 306. Thedisplay apparatus 302 further comprises means for detecting a gaze direction 310. Thedisplay apparatus 302 is arranged to be detachably attached to a portable electronic device 304 and to be communicably coupled with the portable electronic device 304. Furthermore, as shown, the portable electronic device 304 comprises a processor 312 configured to render a context image at a display 314 of the portable electronic device 304. - Referring to
FIG. 4 , illustrated is an exemplary implementation of a display system (such as thedisplay system 100, shown inFIG. 1 ), in accordance with an embodiment of the present disclosure. The display system is shown to include adisplay 402 of a portable electronic device (such as the portableelectronic device 104 ofFIG. 1 ) to render a context image, at least onefocus display 404 of a display apparatus (such as the display apparatus 102 ofFIG. 1 ) to render a focus image, and at least oneoptical combiner 406. Furthermore, at least one focusinglens 408 is positioned on an optical path of projections of the context and focus images. The at least one focusinglens 408 is one of an enlarging or a shrinking (namely, reducing) lens. - Referring to
FIG. 5 , illustrated are steps of amethod 500 of displaying via the display apparatus 102, in accordance with an embodiment of the present disclosure. Atstep 502, a focus image is rendered at, at least one focus display of the display apparatus, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees. Atstep 504, a context image is rendered at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees. Atstep 506, the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene. - The
steps 502 to 506 are only illustrative and other alternatives can also be provided where one or more steps are added, one or more steps are removed, or one or more steps are provided in a different sequence without departing from the scope of the claims herein. For example, themethod 500 further comprises receiving, at the display apparatus, the focus image from the portable electronic device. In an example, wherein the display apparatus further comprises means for detecting a gaze direction, and wherein themethod 500 further comprises detecting, at the display apparatus, a gaze direction, communicating the detected gaze direction to the portable electronic device, obtaining an input image at the portable electronic device, and using the detected gaze direction to determine a region of visual accuracy of the input image, processing the input image to generate the context image and the focus image at the portable electronic device, the context image having a first resolution and the focus image having a second resolution, wherein a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked, the focus image substantially corresponds to the region of visual accuracy of the input image, and the second resolution is higher than the first resolution, and communicating the focus image from the portable electronic device to the display apparatus. In another example, wherein the display apparatus further comprises means for detecting a gaze direction, and wherein themethod 500 further comprises detecting a gaze direction, obtaining an input image at the display apparatus, and using the detected gaze direction to determine a region of visual accuracy of the input image, processing the input image to generate the context image and the focus image at the display apparatus, the context image having a first resolution and the focus image having a second resolution, wherein a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked, the focus image substantially corresponds to the region of visual accuracy of the input image, and the second resolution is higher than the first resolution, and communicating the context image from the display apparatus to the portable electronic device. In one example, themethod 500 further comprises using at least one optical combiner of the display apparatus to combine the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image. In another example, the at least one optical combiner comprises at least one first optical element, and wherein themethod 500 further comprises adjusting a location of the projection of the rendered focus image on the at least one first optical element, the adjusting being performed by controlling at least one first actuator of the at least one optical combiner to move at least one of the at least one focus display with respect to the at least one first optical element of the at least one optical combiner, at least one second optical element of the at least one optical combiner with respect to the at least one first optical element, the at least one second optical element being positioned on an optical path between the at least one first optical element and the at least one focus display, the at least one first optical element of the at least one optical combiner. - Modifications to embodiments of the present disclosure described in the foregoing are possible without departing from the scope of the present disclosure as defined by the accompanying claims. Expressions such as “including”, “comprising”, “incorporating”, “have”, “is” used to describe and claim the present disclosure are intended to be construed in a non-exclusive manner, namely allowing for items, components or elements not explicitly described also to be present. Reference to the singular is also to be construed to relate to the plural.
Claims (15)
1. A display apparatus comprising:
at least one focus display;
a processor coupled to the at least one focus display, wherein the processor is configured to render a focus image at the at least one focus display, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees,
wherein the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled with the portable electronic device, and wherein a processor of the portable electronic device is configured to render a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees,
means for detecting a gaze direction, wherein the processor of the display apparatus is configured to communicate the detected gaze direction to the processor of the portable electronic device; wherein the processor of the portable electronic device is configured to:
(a) obtain an input image, and use the detected gaze direction to determine a region of visual accuracy of the input image;
(b) process the input image to generate the context image and the focus image, the context image having a first resolution and the focus image having a second resolution, wherein:
a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked,
(ii) the focus image substantially corresponds to the region of visual accuracy of the input image, and
(iii) the second resolution is higher than the first resolution; and
(c) communicate the focus image to the processor of the display apparatus; and
wherein the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
2. The display apparatus of claim 1 , wherein the processor of the display apparatus is configured to receive the focus image from the processor of the portable electronic device.
3. (canceled)
4. A display apparatus comprising:
at least one focus display;
a processor coupled to the at least one focus display, wherein the processor is configured to render a focus image at the at least one focus display, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees,
wherein the display apparatus is arranged to be detachably attached to a portable electronic device and to be communicably coupled with the portable electronic device, and wherein a processor of the portable electronic device is configured to render a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220, further comprising means for detecting a gaze direction, wherein the processor of the display apparatus is configured to:
(a) obtain an input image, and use the detected gaze direction to determine a region of visual accuracy of the input image;
(b) process the input image to generate the context image and the focus image, the context image having a first resolution and the focus image having a second resolution, wherein:
(i) a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked,
(ii) the focus image substantially corresponds to the region of visual accuracy of the input image, and
(iii) the second resolution is higher than the first resolution; and
(c) communicate the context image to the processor of the portable electronic device; and
wherein the projection of the rendered context image is combined with the projection of the rendered focus image to create a visual scene.
5. The display apparatus of claim 1 , further comprising at least one optical combiner for combining the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image.
6. The display apparatus of claim 5 , wherein the at least one optical combiner comprises at least one first optical element and at least one first actuator for adjusting a location of the projection of the rendered focus image on the at least one first optical element, wherein the processor of the display apparatus is configured to control the at least one first actuator to move at least one of:
(i) the at least one focus display with respect to the at least one first optical element of the at least one optical combiner,
(ii) at least one second optical element of the at least one optical combiner with respect to the at least one first optical element, the at least one second optical element being positioned on an optical path between the at least one first optical element and the at least one focus display,
(iii) the at least one first optical element of the at least one optical combiner.
7. The display apparatus of claim 6 , further comprising:
at least one focusing lens that is positioned on an optical path between the at least one first optical element and the at least one focus display; and
at least one second actuator for moving the at least one focusing lens with respect to the at least one focus display,
wherein the processor of the display apparatus is configured to control the at least one second actuator to adjust a focus of the projection of the rendered focus image.
8. The display apparatus of claim 1 , wherein the at least one focus display is selected from the group consisting of: a Liquid Crystal Display (LCD), a Light Emitting Diode (LED)-based display, an Organic LED (OLED)-based display, a micro OLED-based display, a Liquid Crystal on Silicon (LCoS)-based display, and a Digital Light Processing (DLP)-based display.
9. The display apparatus of claim 1 , wherein the at least one focus display is implemented by way of at least one projector and at least one projection screen.
10. A method of displaying, via a display apparatus comprising at least one focus display, the display apparatus being detachably attached to a portable electronic device and being communicably coupled with the portable electronic device, the method comprising:
rendering a focus image at the at least one focus display of the display apparatus, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees;
rendering a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees; and
combining the projection of the rendered context image with the projection of the rendered focus image to create a visual scene; and
wherein the display apparatus further comprises means for detecting a gaze direction, and wherein the method further comprises:
detecting, at the display apparatus, a gaze direction;
communicating the detected gaze direction to the portable electronic device;
obtaining an input image at the portable electronic device, and using the detected gaze direction to determine a region of visual accuracy of the input image;
processing the input image to generate the context image and the focus image at the portable electronic device, the context image having a first resolution and the focus image having a second resolution, wherein:
(i) a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked,
(ii) the focus image substantially corresponds to the region of visual accuracy of the input image, and
(iii) the second resolution is higher than the first resolution; and
communicating the focus image from the portable electronic device to the display apparatus.
11. The method of claim 10 , further comprising receiving, at the display apparatus, the focus image from the portable electronic device.
12. (canceled)
13. A method of displaying, via a display apparatus comprising at least one focus display, the display apparatus being detachably attached to a portable electronic device and being communicably coupled with the portable electronic device, the method comprising:
rendering a focus image at the at least one focus display of the display apparatus, wherein an angular width of a projection of the rendered focus image ranges from 5 degrees to 60 degrees;
rendering a context image at a display of the portable electronic device, wherein an angular width of a projection of the rendered context image ranges from 40 degrees to 220 degrees; and
combining the projection of the rendered context image with the projection of the rendered focus image to create a visual scene; and wherein the display apparatus further comprises means for detecting a gaze direction, and wherein the method further comprises:
detecting a gaze direction;
obtaining an input image at the display apparatus, and using the detected gaze direction to determine a region of visual accuracy of the input image;
processing the input image to generate the context image and the focus image at the display apparatus, the context image having a first resolution and the focus image having a second resolution, wherein:
(i) a region of the context image that substantially corresponds to the region of visual accuracy of the input image is masked,
(ii) the focus image substantially corresponds to the region of visual accuracy of the input image, and
(iii) the second resolution is higher than the first resolution; and
communicating the context image from the display apparatus to the portable electronic device.
14. The method of claim 10 , further comprising using at least one optical combiner of the display apparatus to combine the projection of the rendered context image with the projection of the rendered focus image in a manner that the projection of the rendered focus image substantially overlaps the projection of the masked region of the rendered context image.
15. The method of claim 14 , wherein the at least one optical combiner comprises at least one first optical element, and wherein the method further comprises adjusting a location of the projection of the rendered focus image on the at least one first optical element, the adjusting being performed by controlling at least one first actuator of the at least one optical combiner to move at least one of:
(i) the at least one focus display with respect to the at least one first optical element of the at least one optical combiner,
(ii) at least one second optical element of the at least one optical combiner with respect to the at least one first optical element, the at least one second optical element being positioned on an optical path between the at least one first optical element and the at least one focus display,
(iii) the at least one first optical element of the at least one optical combiner.
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/623,052 US9990037B1 (en) | 2016-12-01 | 2017-06-14 | Display apparatus and method using portable electronic device |
CN201780072790.6A CN109997067B (en) | 2016-12-01 | 2017-11-27 | Display apparatus and method using portable electronic device |
PCT/FI2017/050824 WO2018100236A1 (en) | 2016-12-01 | 2017-11-27 | Display apparatus and method using portable electronic device |
US15/907,914 US10082672B2 (en) | 2016-12-01 | 2018-02-28 | Display apparatus and method of displaying using electromechanical faceplate |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/366,424 US9711072B1 (en) | 2016-12-01 | 2016-12-01 | Display apparatus and method of displaying using focus and context displays |
US15/623,052 US9990037B1 (en) | 2016-12-01 | 2017-06-14 | Display apparatus and method using portable electronic device |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/366,424 Continuation-In-Part US9711072B1 (en) | 2016-12-01 | 2016-12-01 | Display apparatus and method of displaying using focus and context displays |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/907,914 Continuation-In-Part US10082672B2 (en) | 2016-12-01 | 2018-02-28 | Display apparatus and method of displaying using electromechanical faceplate |
Publications (2)
Publication Number | Publication Date |
---|---|
US9990037B1 US9990037B1 (en) | 2018-06-05 |
US20180157322A1 true US20180157322A1 (en) | 2018-06-07 |
Family
ID=60654988
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/623,052 Active US9990037B1 (en) | 2016-12-01 | 2017-06-14 | Display apparatus and method using portable electronic device |
Country Status (3)
Country | Link |
---|---|
US (1) | US9990037B1 (en) |
CN (1) | CN109997067B (en) |
WO (1) | WO2018100236A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102656425B1 (en) * | 2016-12-07 | 2024-04-12 | 삼성전자주식회사 | Electronic apparatus and method for displaying an image |
CN112578564B (en) * | 2020-12-15 | 2023-04-11 | 京东方科技集团股份有限公司 | Virtual reality display equipment and display method |
CN114019679A (en) * | 2021-10-22 | 2022-02-08 | 小派科技(上海)有限责任公司 | Optical module/system, display device, head-mounted display equipment and display system |
CN113933998A (en) * | 2021-10-22 | 2022-01-14 | 小派科技(上海)有限责任公司 | Optical module/system, display device, head-mounted display equipment and display system |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3524408B2 (en) | 1998-01-06 | 2004-05-10 | キヤノン株式会社 | Observation optical system and optical apparatus having the same |
US7860382B2 (en) * | 2006-10-02 | 2010-12-28 | Sony Ericsson Mobile Communications Ab | Selecting autofocus area in an image |
US8957835B2 (en) * | 2008-09-30 | 2015-02-17 | Apple Inc. | Head-mounted display apparatus for retaining a portable electronic device with display |
JP5671842B2 (en) | 2010-06-03 | 2015-02-18 | 株式会社ニコン | Image processing apparatus and imaging apparatus |
US9551873B2 (en) | 2014-05-30 | 2017-01-24 | Sony Interactive Entertainment America Llc | Head mounted device (HMD) system having interface with mobile computing device for rendering virtual reality content |
TWI688789B (en) | 2014-11-20 | 2020-03-21 | 美商英特爾股份有限公司 | Virtual image generator and method to project a virtual image |
DE112016000728T5 (en) * | 2015-02-12 | 2017-10-26 | Google Inc. | Combine a narrow field of view and high resolution display with a medium resolution wide angle display |
US10417832B2 (en) | 2015-03-11 | 2019-09-17 | Facebook Technologies, Llc | Display device supporting configurable resolution regions |
WO2016187352A1 (en) * | 2015-05-18 | 2016-11-24 | Daqri, Llc | Threat identification system |
-
2017
- 2017-06-14 US US15/623,052 patent/US9990037B1/en active Active
- 2017-11-27 WO PCT/FI2017/050824 patent/WO2018100236A1/en active Application Filing
- 2017-11-27 CN CN201780072790.6A patent/CN109997067B/en active Active
Also Published As
Publication number | Publication date |
---|---|
US9990037B1 (en) | 2018-06-05 |
CN109997067A (en) | 2019-07-09 |
WO2018100236A1 (en) | 2018-06-07 |
CN109997067B (en) | 2021-12-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9711072B1 (en) | Display apparatus and method of displaying using focus and context displays | |
US9711114B1 (en) | Display apparatus and method of displaying using projectors | |
US9990037B1 (en) | Display apparatus and method using portable electronic device | |
US10382699B2 (en) | Imaging system and method of producing images for display apparatus | |
US10602033B2 (en) | Display apparatus and method using image renderers and optical combiners | |
EP3548955B1 (en) | Display apparatus and method of displaying using image renderers and optical combiners | |
US20200049946A1 (en) | Display apparatus and method of displaying using gaze prediction and image steering | |
KR101690646B1 (en) | Camera driving device and method for see-through displaying | |
WO2019151817A1 (en) | Apparatus and method for tracking a focal point in a head mounted display system | |
WO2018100242A1 (en) | Display apparatus and method of displaying using optical combiners and context and focus image renderers | |
JP6509101B2 (en) | Image display apparatus, program and method for displaying an object on a spectacle-like optical see-through type binocular display | |
CN112805755A (en) | Information processing apparatus, information processing method, and recording medium | |
CN110895433B (en) | Method and apparatus for user interaction in augmented reality | |
EP3548956B1 (en) | Imaging system and method of producing context and focus images | |
EP3330773A1 (en) | Display apparatus and method of displaying using context display and projectors | |
GB2533573A (en) | Image processing method and device | |
CN112236711A (en) | Apparatus and method for image display | |
US20200033613A1 (en) | Display apparatus and method of displaying using curved optical combiner | |
JP2020167657A (en) | Image processing apparatus, head-mounted display, and image display method | |
US12058452B2 (en) | Image blending | |
US20220252884A1 (en) | Imaging system, display device, imaging device, and control method for imaging system | |
JP2024005644A (en) | Focusing control device, display device, and image transmission device | |
JP2023143437A (en) | Control device, control method, and control program | |
CN117170602A (en) | Electronic device for displaying virtual object |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: VARJO TECHNOLOGIES OY, FINLAND Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RAINISTO, ROOPE;REEL/FRAME:042710/0867 Effective date: 20170608 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY Year of fee payment: 4 |