WO2019239162A1 - Dispositif portatif pour commander un grossissement numérique sur un dispositif d'affichage portable - Google Patents

Dispositif portatif pour commander un grossissement numérique sur un dispositif d'affichage portable Download PDF

Info

Publication number
WO2019239162A1
WO2019239162A1 PCT/GB2019/051688 GB2019051688W WO2019239162A1 WO 2019239162 A1 WO2019239162 A1 WO 2019239162A1 GB 2019051688 W GB2019051688 W GB 2019051688W WO 2019239162 A1 WO2019239162 A1 WO 2019239162A1
Authority
WO
WIPO (PCT)
Prior art keywords
computing device
portable computing
image
wearable device
magnification
Prior art date
Application number
PCT/GB2019/051688
Other languages
English (en)
Inventor
Stephen Lloyd Frederick HICKS
Noah Aaron RUSSELL
Original Assignee
Oxsight Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from GBGB1809905.1A external-priority patent/GB201809905D0/en
Priority claimed from GBGB1809904.4A external-priority patent/GB201809904D0/en
Priority claimed from GBGB1819144.5A external-priority patent/GB201819144D0/en
Priority claimed from GBGB1819145.2A external-priority patent/GB201819145D0/en
Application filed by Oxsight Ltd filed Critical Oxsight Ltd
Priority to CN201980043476.4A priority Critical patent/CN112313731A/zh
Priority to GB2020076.2A priority patent/GB2588055A/en
Publication of WO2019239162A1 publication Critical patent/WO2019239162A1/fr

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G3/00—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes
    • G09G3/001—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background
    • G09G3/003—Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background to produce spatial visual effects
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/163—Wearable computers, e.g. on a belt
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
    • G06F1/16—Constructional details or arrangements
    • G06F1/1613—Constructional details or arrangements for portable computers
    • G06F1/1633—Constructional details or arrangements of portable computers not specific to the type of enclosures covered by groups G06F1/1615 - G06F1/1626
    • G06F1/1684—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
    • G06F1/1694—Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0485—Scrolling or panning
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/39—Control of the bit-mapped memory
    • G09G5/391—Resolution modifying circuits, e.g. variable screen formats
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0112—Head-up displays characterised by optical features comprising device for genereting colour display
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0118—Head-up displays characterised by optical features comprising devices for improving the contrast of the display / brillance control visibility
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0101—Head-up displays characterised by optical features
    • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/0179—Display position adjusting means not related to the information to be displayed
    • G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04806—Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/0606—Manual adjustment
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/066—Adjustment of display parameters for control of contrast
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/0666—Adjustment of display parameters for control of colour parameters, e.g. colour temperature
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/06—Adjustment of display parameters
    • G09G2320/0673—Adjustment of display parameters for control of gamma adjustment, e.g. selecting another gamma curve
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2320/00—Control of display operating conditions
    • G09G2320/08—Arrangements within a display terminal for setting, manually or automatically, display parameters of the display terminal
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/04—Changes in size, position or resolution of an image
    • G09G2340/045—Zooming at least part of an image, i.e. enlarging it or shrinking it
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00—Aspects of interface with display user

Definitions

  • the present disclosure pertains to a system, device and method thereof for controlling digital magnification on a portable display, such as that of a smart glass.
  • Human vision is capable of perceiving a very broad range of scales from the largest field of view such as an entire room, to very narrow such as reading letters or concentrating on smaller objects.
  • Wearable display systems such as augmented reality (AR) or virtual reality (VR) devices typically have a fixed field of view that is determined by the resolution of imaging system (e.g. LCD, OLED, or LCoS display panel) and optics that has been devised to bring the image into focus (in the case of a near-eye display). These mechanisms necessarily limit the amount of information that can be displayed at any one time.
  • AR augmented reality
  • VR virtual reality
  • magnification, or zoom can be used to increase visibility of small objects (i.e. magnify or zoom in), or to expand the field of view (i.e. minify or zoom out).
  • buttons can be difficult to press for elderly people or those with reduced sensitivity in the fingers.
  • Pinch and zoom technique is only intuitive for a tactile display where visual feedback of the scale of motion of the fingers is directly related to resizing of the image.
  • large hand gestures are impractical in crowded situations. They are obvious to those around the user and may be a source of embarrassment. Hand gestures are also challenging to accurately detect by mechanical or computer systems in difficult lighting situations.
  • the present disclosure pertains to a device and method thereof for controlling digital magnification on a portable display, such as that of a smart glass.
  • smart glass is meant wearable computer glasses (or‘spectacles’) that provide visual information in a user’s field of view in addition to that which the user is able to view substantially directly (either with or without an intermediate optical element such as a lens, which may be substantially transparent or partially transparent).
  • the provision of visual information in addition to that which the user views substantially directly may be by superimposing information onto the user’s field of view, for example on a display element in the user’s field of view (which display element may be substantially transparent, at least partially transparent or opaque).
  • the display may be a substantially opaque LED or LCD display of the type used in mobile telephones or partially transparent.
  • an image may be projected onto the display from a light source in the form or a projector device, for example of the type used in head-up display (HUD) displays or augmented reality (AR) overlays and the reflected image viewed by the user.
  • HUD head-up display
  • AR augmented reality
  • a smart glass system arranged to permit a visually impaired user to control magnification of an image provided by the system according to their eye condition, comprising:
  • a portable computing device comprising a motion sensor
  • a smart glass based wearable device comprising a display portion, the display portion being provided in a field of view of the user;
  • an image capture device wherein the portable computing device is operatively coupled with the smart glass based wearable device
  • system being configured to display on the display portion an image corresponding to at least a portion of an image captured by the image capture device
  • system is configured to detect rotational movement of the portable computing device in the hand of the user by means of the motion sensor, wherein magnification of the image displayed on the wearable device is controlled based on the rotational movement of the portable computing device.
  • the user may endeavour to optimise the magnification of a scene as viewed using the smart glass based wearable device.
  • the system may be configured wherein the user may adjust the magnification in substantially real time.
  • the system may be configured to permit optimisation of the magnification in substantially real time, the magnification being changed dynamically as the user manipulates the portable computing device in their hand.
  • the image capture device may comprise a video image capture device.
  • the image capture device may comprise at least one CMOS image capture device and/or at least one CCD image capture device. Other image capture devices may be useful.
  • the wearable device may generate a substantially real time stream of images captured by the image capture device.
  • the image capture device is configured to capture a scene having at least a portion in a field of view of a person wearing the wearable device, the system being configured to display on the display portion of the wearable device at least a portion of the scene captured by the image capture device corresponding to the location of the display in the field of view of the person wearing the wearable device.
  • a zoomed image (or 1 :1 scale image with no magnification or demagnification with respect to the remainder of the user’s field of view) may appear inset in the user’s field of view, e.g. to one side or corner of the field of view, or in a portion away from an edge of the field of view. It is to be understood that by field of view is meant the view in a direction a user is facing.
  • reference to the“field of view of a person wearing the wearable device” is to be understood to be with respect to a person wearing the wearable device such that the display portion is in their field of view, optionally their field of view when looking substantially directly ahead, optionally their field of view with their eyes directed in a prescribed direction, the direction being any prescribed direction from directly upwards (a ⁇ 2 o’clock’ direction), directly downwards (a‘6 o’clock direction’) or any prescribed direction from 12 o’clock clockwise around to 12 o’clock.
  • the motion sensor comprises a gyroscope that is positioned in a manner such that upon actuation of said portable computing device, orientation of the portable computing device is determined, and upon hand rotation of said portable computing device, magnification of the image displayed on the wearable device is controlled based on rotational velocity computed using said gyroscope.
  • the motion sensor comprises an accelerometer and is arranged to transmit magnitude of linear acceleration of the portable computing device relative to gravity.
  • the motion sensor comprises a magnetometer that determines and transmits instantaneous orientation of the portable computing device relative to Earth’s magnetic field.
  • respective outputs from said gyroscope, said accelerometer, and said magnetometer are fused to yield the orientation and motion of the portable computing device in any direction.
  • the portable computing device is actuated by pressing of a button present in/on said computing device, wherein the magnification is controlled only during the time the button is kept pressed.
  • clockwise hand rotation results in zooming in of the image
  • anti clockwise hand rotation results in zooming out of the image
  • clockwise hand rotation results in zooming out of the image
  • anti-clockwise hand rotation results in zooming in of the image.
  • the extent of hand rotation is proportional to the extent of magnification.
  • the orientation of the portable computing device is determined based on fusion of positional data from one or more components of the motion sensor, the one or more components comprising at least an accelerometer.
  • magnification operation when the magnification operation is paused, user of said wearable device can view said magnified image and/or pan around said magnified image in X and Y axis and/or scroll around said magnified image.
  • an absolute position of said portable computing device is configured to be indicative of level of zoom.
  • the magnification operation comprises any or a combination of stepped zooming or dynamic zooming.
  • the wearable device comprises the image capture device.
  • the image capture device may be an integral part of the wearable device.
  • the image capture device may be arranged to be sufficiently forward facing so as to include at least a portion of a user’s field of view.
  • the image capture device may be arranged to be substantially fully forward facing.
  • the image capture device is provided external to the wearable device, the image capture device being operatively coupled to the wearable device.
  • the image capture device is operatively coupled to the wearable device by means of a wireless connection or a wired connection.
  • the motion sensor is provided by an inertial measurement unit (IMU).
  • IMU inertial measurement unit
  • the IMU may comprise a gyroscope, an accelerometer and/or a magnetometer as described above.
  • the method comprising controlling magnification of the image displayed on the wearable device based on the rotational movement of the portable computing device.
  • the method comprises capturing by means of the image capture device a scene having at least a portion in a field of view of a person wearing the wearable device, the method comprising displaying on the display portion of the wearable device at least a portion of the scene captured by the image capture device.
  • the person wearing the wearable device will see the image captured by the image capture device within their field of view.
  • displaying on the display portion of the wearable device at least a portion of the scene comprises displaying on the display portion at least a portion of the scene corresponding to the location of the display in the field of view of the person wearing the wearable device.
  • the image displayed by the display portion occupies a portion and not the whole of the field of view of the user.
  • the image displayed is substantially continuous with a remainder of a field of view of the user such that the image displayed appears to be superimposed upon or part of the remainder of the field of view.
  • the display portion may be at least partially transparent, allowing the user to see objects in the portion of the field of view occupied by the display portion through the display portion as well as information displayed on the display portion by the system.
  • the motion sensor comprises a gyroscope sensor, the method comprising: receiving from the gyroscope sensor a change in gyroscope value indicative of extent of hand rotation of said portable computing device;
  • said method further comprises the step of:
  • the magnification control signal increases center crop image so as to enlarge display configured in the wearable device.
  • the magnification control signal reduces center crop image so as to enlarge display configured in the wearable device.
  • the portable computing device comprising a motion sensor; the portable computing device being arranged to be operatively coupled with a smart glass based wearable device of the system, the smart glass based wearable device comprising a display portion, the display portion being provided in a field of view of the user, the smart glass system further comprising an image capture device, the system being configured to display on the display portion of the wearable device an image corresponding to at least a portion of an image captured by the image capture device, wherein the system is configured to detect rotational movement of the portable computing device in the hand of the user by means of the motion sensor, wherein magnification of the
  • a smart glass based wearable device arranged to be operatively coupled with the portable computing device of the preceding aspect, the system being arranged to permit a visually impaired user to control magnification of an image provided by the system according to their eye condition.
  • the present disclosure relates to portable computing device (interchangeably referred to as hand-held device or zoom-controlling computing device hereinafter) that is operatively coupled with a smart glass based wearable device (interchangeably referred to as portable display), said portable computing device comprising an inertial measurement unit (IMU) having a gyroscope that is positioned in a manner such that upon actuation of said portable computing device, orientation of the portable computing device is determined, and upon hand rotation of said portable computing device, magnification of image displayed on the wearable device is controlled based on rotational velocity computed using said gyroscope.
  • IMU inertial measurement unit
  • the IMU can further include an accelerometer to transmit magnitude of linear acceleration of the portable computing device relative to gravity.
  • the IMU can further include a magnetometer that can determine and transmit instantaneous orientation of the portable computing device relative to Earth’s magnetic field.
  • respective outputs from the gyroscope, the accelerometer, and the magnetometer are fused to yield the orientation and motion of the portable computing device in any direction.
  • the portable computing device can be actuated by pressing of a button present in/on said computing device, wherein the magnification is controlled only during the time the button is kept pressed.
  • clockwise hand rotation can result in zooming in of the image
  • anti-clockwise hand rotation results in zooming out of the image
  • the extent of hand rotation can be proportional to the extent of magnification.
  • the orientation of the portable computing device can be determined based on fusion of positional data from one or more components of the IMU, said one or more components comprising at least an accelerometer.
  • magnification operation when the magnification operation is paused, user of said wearable device can view said magnified image and/or pan around said magnified image in X and Y axis and/or scroll around said magnified image.
  • an absolute position of said portable computing device can be configured to be indicative of level of zoom.
  • magnification operation can include any or a combination of stepped zooming or dynamic zooming.
  • the present disclosure further relates to a method of controlling magnification operation on an image being displayed in a smart glass based wearable device by a portable computing device, said method comprising the step of: receiving, at the portable computing device, from a gyroscope sensor configured in the portable computing device, a change in gyroscope value indicative of extent of hand rotation of said portable computing device; determining, at the portable computing device, using an accelerometer configured in the portable computing device, orientation of the portable computing device; and generating, from the portable computing device, a magnification control signal to said glass based wearable device based on the change in gyroscope value and the determined orientation.
  • the method can further include steps of receiving new gyroscope values as part of the change in gyroscope value; smoothing, using a sliding window fdter, the received new gyroscope values; normalizing said smoothened gyroscope values; and accumulating the normalized gyroscope values to indicate the extent of hand rotation of said portable computing device.
  • the magnification control signal increases center crop image so as to enlarge display configured in the wearable device.
  • the magnification control signal reduces center crop image so as to enlarge display configured in the wearable device.
  • the portable computing device can include an inertial measurement unit (IMU) having a gyroscope that is positioned in a manner such that upon actuation of the portable computing device, orientation of the portable computing device is determined, and upon hand rotation of the portable computing device, magnification of image displayed on the wearable device is controlled based on rotational velocity computed using the gyroscope.
  • IMU inertial measurement unit
  • FIGs. 1 and 2 illustrate exemplary representations of the proposed device in accordance with an embodiment of the present disclosure.
  • FIG. 3 illustrates an exemplary flow diagram to enable zooming of captured videos/images in accordance with an embodiment of the present disclosure.
  • FIG. 4 illustrates an exemplary flow diagram to enable head pan operation during pause action during magnification in accordance with an embodiment of the present disclosure.
  • Embodiments of the present disclosure include various steps, which will be described below.
  • the steps may be performed by hardware components or may be embodied in machine- executable instructions, which may be used to cause a general-purpose or special-purpose processor programmed with the instructions to perform the steps.
  • steps may be performed by a combination of hardware, software, firmware and/or by human operators.
  • Embodiments of the present disclosure may be provided as a computer program product, which may include a machine-readable storage medium tangibly embodying thereon instructions, which may be used to program a computer (or other electronic devices) to perform a process.
  • the machine-readable medium may include, but is not limited to, fixed (hard) drives, magnetic tape, floppy diskettes, optical disks, compact disc read-only memories (CD-ROMs), and magneto-optical disks, semiconductor memories, such as ROMs, PROMs, random access memories (RAMs), programmable read-only memories (PROMs), erasable PROMs (EPROMs), electrically erasable PROMs (EEPROMs), flash memory, magnetic or optical cards, or other type of media/machine-readable medium suitable for storing electronic instructions (e.g., computer programming code, such as software or firmware).
  • Various methods described herein may be practiced by combining one or more machine-readable storage media containing the code according to the present disclosure with appropriate standard computer hardware to execute the code contained therein.
  • An apparatus for practicing various embodiments of the present disclosure may involve one or more computers (or one or more processors within a single computer) and storage systems containing or having network access to computer program(s) coded in accordance with various methods described herein, and the method steps of the disclosure could be accomplished by modules, routines, subroutines, or subparts of a computer program product.
  • the present disclosure relates to a device (also interchangeably referred to as apparatus or system) that permits rapid, accurate, and finely tuned zooming of a live image with a simple physical behaviour that is pronounced of common actions, such as that used when one“turns up the volume” on a traditional sound system.
  • a device also interchangeably referred to as apparatus or system
  • zooming of a live image with a simple physical behaviour that is pronounced of common actions, such as that used when one“turns up the volume” on a traditional sound system.
  • the present disclosure provides an electronic system that contains an inertial measurement device (IMU) having a gyroscope and an accelerometer (it may also alternatively or additionally include a magnetometer), wherein during implementation/operation, a user can press and hold a button on the proposed device and then rotate his/her hand as if controlling the volume knob on an audio device. Rotations to the right cause the image to be magnified, whereas rotations to the left cause the image to be minified. These directions can of course, if desired, be reversed, and all such variations are well within the scope of the present invention.
  • the degree of rotation determines the degree to which the image is resized. A small rotation makes a small change to the image, whereas a larger rotation causes the image to be changed by a greater magnitude.
  • the proposed system can be activated by a button press, wherein at a first press, orientation of the portable device/handset can be calculated.
  • the orientation can be determined based on fusion of positional data from one or more components of the IMU.
  • the accelerometer can then give real-time value of the device/handset’s position relative to gravity.
  • the accelerometer can also be configured to gives/transmit/send the magnitude of any linear acceleration in three dimensions.
  • the gyroscope on the other hand, can be configured to indicate instantaneous rotational velocity in three dimensions.
  • the magnetometer can be configured to determine and transmit instantaneous orientation of the proposed device/handset relative to Earth’s magnetic field (i.e. a compass).
  • initial orientation of the handset becomes set zero. Any rotation about a defined axis of the handset can be interpreted as an increase or a decrease of the level of zoom.
  • axis of rotation can be defined to be along the length of the device, which is the same axis as the wrist.
  • a clockwise roll increases zoom, and where anticlockwise roll decreases zoom.
  • zoom control can be only active while the button is depressed. When the button is released, the level of zoom can either return to its previous pre activation level, or it can remain at the new zoom level.
  • ratio of zoom level per degree of rotation can be adjusted to allow a greater degree of zoom accuracy (e.g. larger rotation equates to small zoom steps), or low fidelity and greater speed (e.g. smaller rotations equates to large zoom steps).
  • the proposed device allows the speed of zoom to be very fast. For instance, turning the wrist 45 degrees could be sufficient to achieve maximum zoom.
  • a button would have to be pressed many times to achieve the same effect.
  • the proposed zoom enabling product offers the user a much finer resolution of zoom control over individually set zoom steps using buttons. This can allow a person to magnify objects so that they are very easy to see. This is particularly important for people with impaired vision, for instance, it may be necessary to magnify text to a particular size in order to see it. Buttons would have to have predefined steps and hence it might be challenging to find the right level that way. Additionally, using the proposed rotation product/technique can allow the zoom level to move smoothly from one level to another, which is more comfortable and less disorienting than if the user experiences large disconnected and discrete jumps from one zoom level to another.
  • zoom level steps can be configured for different eye conditions.
  • proposed device of the present disclosure can also enable functions/operations of any or a combination of pause, zoom and pan.
  • the proposed device can include an architecture that can be built on top of rotational zoom control wherein when the button is pressed, a live image is captured and paused and the user can employ the rotational zoom control as described above.
  • the pause operation/function can allow a person with severely impaired sight to spend a longer time viewing the scene. This is necessary as often sight impaired people require a longer time to see certain objects, for example text in the environment, such as train times.
  • the user zooms in to the paused image he/she can then move his/her head to pan around the image in X and Y.
  • the headset such as a smart glass that includes a camera and a display can also include an IMU that is similar to the one in the proposed handheld device. Rotations or translations of the head can be converted into pan signals in X and Y, and through this method, an individual is able to scroll around the image, as well as zoom in and out.
  • the absolute position of the handset/smart glass can be taken as indicating the level of zoom. Hence, a person can learn that a rotation to, say, 2 o’clock will increase the magnitude of zoom to 2x.
  • aspects of the present disclosure can also enable dynamic zoom (continuous not stepped) delivered quickly and intuitively means that the operator experiences a shorter feedback loop so as to avoid disorientation, allow fine tuning (particularly useful when reading text), and enable quick correction (zoom in, out and then relocate zoom) to find object being searched for.
  • FIGs. 1 and 2 illustrate exemplary representation of the proposed device in accordance with an embodiment of the present disclosure, wherein the device can either be physically connected to the smart glass/wearable device or can be wirelessly coupled through Bluetooth or can be mounted onto the frame of the smart glass/wearable device, or any other configuration, all of which are well within the scope of the present disclosure.
  • FIG. 1 (a) a system 10 is provided with a smart glass based wearable device 160 that has a display screen 162 and an image capture device in the form of a video camera 163.
  • FIG. 1 (b) shows a corresponding embodiment in which the image capture device of the system 10A is not provided integral to the wearable device 160. Rather, it may be coupled to the device 160 via a wireless connection. In some embodiments the image capture device may be coupled by means of a wired connection in addition or instead.
  • the display screen 162 is a transparent waveguide with diffractive optics arranged to direct an image from an organic light emitting diode (OLED) micro display, into the user’s eye.
  • OLED organic light emitting diode
  • CMOS complementary metal oxide semiconductor
  • an electronic system/product 100 can include an inertial measurement device (IMU) 102 having a gyroscope 104 and an accelerometer 106 (it may also alternatively or additionally include a magnetometer 108), wherein during implementation/operation, a user can press and hold a button 150 on the proposed device 100, and then rotate his/her hand as if controlling the dial on a volume control. Rotations to the right cause the image presented on the display 162 of the coupled smart glass/wearable device 160 to be magnified, whereas rotations to the left cause the image to be minified. These directions can of course, if desired, be reversed, and all such variations are well within the scope of the present invention.
  • IMU inertial measurement device
  • the degree of rotation dictates the amount that the image is resized. A small rotation makes a small change to the image, whereas a larger rotation causes the image to be changed by a greater magnitude.
  • the system/product 100 in combination with the smart glass/wearable device 160 may be referred to in combination as a smart glass system or smart glasses system or‘system’ 10, 10A.
  • axis of rotation can be defined to be along the length of the device 100, which is the same axis as the wrist.
  • a clockwise roll increases zoom, and where anticlockwise roll decreases zoom.
  • zoom control can be only active while the button 150 is depressed. When the button 150 is released, the level of zoom can either return to its previous pre-activation level, or it can remain at the new zoom level.
  • ratio of zoom level per degree of rotation can adjusted to allow a greater degree of zoom accuracy (e.g. larger rotation equates to small zoom steps), or low fidelity and greater speed (e.g. smaller rotations equates to large zoom steps).
  • FIG. 3 illustrates an exemplary flow diagram to enable zooming of captured videos/images in accordance with an embodiment of the present disclosure.
  • a smart glass/wearable device/portable display can be operatively coupled with the proposed zoom-enabling computing device (or simply referred to as hand-held device or computing device or portable device), wherein the computing/hand-held device can listen to a sensor that is configured therein, based on which at step 304, the zoom-enabling computing device can receive a change in y-axis gyroscope value (from a gyroscope sensor), output of which can be smoothened, for instance, with a sliding window fdter, at step 306.
  • the output of gyroscope values can be normalized, followed by, at step 310, accumulation of the gyro values.
  • the zoom-enabling computing device can be rotated in a defined direction (clockwise or anti-clockwise) so as to, at step 312, increase center crop image and perform zoom-out operation to enlarge the display of the wearable device, and at step 314, reduce center crop image so as to zoom-in and enlarge to display.
  • a key up instruction can be received by the zoom-enabling computing device, based on which, at step 318, zoom values can return back to the original setting.
  • step 402 during the magnification/zooming operation, when the operation is paused through key down on the button of the portable computing device, at step 404, on the portable computing device (handset), the portable computing device can listen to the sensor that is configured in in the (zoom-enabling) portable computing device, based on which at step 406, the zoom-enabling computing device receives a change in y-axis gyroscope value, output of which is smoothed with a sliding window filter at step 408.
  • the output can be normalized, followed by, at step 412, accumulating the gyro values.
  • the zoom enabling computing device can be rotated in a defined direction (clockwise or anti-clockwise) so as to, at step 414, increase center crop image so as to zoom-out and enlarge the fill screen, and at step 416, reduce center crop image so as to zoom-in and enlarge to fill screen.
  • a key up instruction is received by the zoom-enabling computing device, based on which at step 420, zoom values can return back to the original setting.
  • step 450 a signal from the sensor is waited for, based on which at step 452, changes to X and Y axes gyroscope values are received, and at step 454, such received gyroscope values are smoothed, and at step 456, X and Y gyro values are mapped into 2D X and Y pan values, post which, at step 458, the enlarged image across the view window can be translocated.
  • a portable computing device operatively coupled with a smart glass based wearable device, said portable computing device comprising an inertial measurement unit (IMU) having a gyroscope that is positioned in a manner such that upon actuation of said portable computing device, orientation of the portable computing device is determined, and upon hand rotation of said portable computing device, magnification of image displayed on the wearable device is controlled based on rotational velocity computed using said gyroscope.
  • IMU inertial measurement unit
  • said IMU further comprises an accelerometer to transmit magnitude of linear acceleration of the portable computing device relative to gravity.
  • magnification operation comprises any or a combination of stepped zooming or dynamic zooming.
  • a method of controlling magnification operation on an image being displayed in a smart glass based wearable device by a portable computing device comprising the step of: receiving, at the portable computing device, from a gyroscope sensor configured in the portable computing device, a change in gyroscope value indicative of extent of hand rotation of said portable computing device;
  • Coupled to and“coupled with” are also used euphemistically to mean“communicatively coupled with” over a network, where two or more devices are able to exchange data with each other over the network, possibly via one or more intermediary device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

La présente invention concerne un système à verre intelligent (10) conçu pour permettre à un utilisateur malvoyant de commander le grossissement d'une image fournie par le système selon son état de santé oculaire, comprenant : un dispositif informatique portable (100) comprenant un capteur de mouvement; un dispositif vestimentaire reposant sur un verre intelligent (160) comprenant une partie d'affichage (162), la partie d'affichage (162) étant disposée dans un champ de vision de l'utilisateur; et un dispositif de capture d'image (163), le dispositif informatique portable (100) étant couplé fonctionnellement au dispositif vestimentaire reposant sur un verre intelligent (160), le système (10) étant configuré pour afficher sur la partie d'affichage (162) une image correspondant à au moins une partie d'une image capturée par le dispositif de capture d'image (163), le système (10) étant configuré pour détecter un mouvement de détection du dispositif informatique portable (100) dans la main de l'utilisateur au moyen du capteur de mouvement (102), le grossissement de l'image affichée sur le dispositif vestimentaire (160) étant commandé sur la base du mouvement de rotation du dispositif informatique portable (100).
PCT/GB2019/051688 2018-06-16 2019-06-17 Dispositif portatif pour commander un grossissement numérique sur un dispositif d'affichage portable WO2019239162A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201980043476.4A CN112313731A (zh) 2018-06-16 2019-06-17 用于在便携式显示器上控制数字放大率的手持装置
GB2020076.2A GB2588055A (en) 2018-06-16 2019-06-17 Hand held device for controlling digital magnification on a portable display

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
GBGB1809905.1A GB201809905D0 (en) 2018-06-16 2018-06-16 Hand held device for controlling digital magnification on a portable display
GB1809904.4 2018-06-16
GBGB1809904.4A GB201809904D0 (en) 2018-06-16 2018-06-16 System and method for modifying image enhancement parameters for a portable display
GB1809905.1 2018-06-16
GB1819145.2 2018-11-24
GB1819144.5 2018-11-24
GBGB1819144.5A GB201819144D0 (en) 2018-11-24 2018-11-24 System and method for modifying image enhancement parameters for a portable display
GBGB1819145.2A GB201819145D0 (en) 2018-11-24 2018-11-24 Hand held device for controlling digital magnification on a portable display

Publications (1)

Publication Number Publication Date
WO2019239162A1 true WO2019239162A1 (fr) 2019-12-19

Family

ID=66998444

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/GB2019/051687 WO2019239161A1 (fr) 2018-06-16 2019-06-17 Système et procédé de modification de paramètres d'amélioration d'image pour un dispositif d'affichage portable
PCT/GB2019/051688 WO2019239162A1 (fr) 2018-06-16 2019-06-17 Dispositif portatif pour commander un grossissement numérique sur un dispositif d'affichage portable

Family Applications Before (1)

Application Number Title Priority Date Filing Date
PCT/GB2019/051687 WO2019239161A1 (fr) 2018-06-16 2019-06-17 Système et procédé de modification de paramètres d'amélioration d'image pour un dispositif d'affichage portable

Country Status (3)

Country Link
CN (2) CN112424729A (fr)
GB (2) GB2588055A (fr)
WO (2) WO2019239161A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112733624A (zh) * 2020-12-26 2021-04-30 电子科技大学 室内密集场景的人流密度检测方法、系统存储介质及终端

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110221656A1 (en) * 2010-02-28 2011-09-15 Osterhout Group, Inc. Displayed content vision correction with electrically adjustable lens
US20170336882A1 (en) * 2016-05-17 2017-11-23 Google Inc. Virtual/augmented reality input device

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090262074A1 (en) * 2007-01-05 2009-10-22 Invensense Inc. Controlling and accessing content using motion processing on mobile devices
JP2009140107A (ja) * 2007-12-04 2009-06-25 Sony Corp 入力装置及び制御システム
JP4725818B2 (ja) * 2009-02-20 2011-07-13 ソニー株式会社 入力装置および方法、情報処理システム、並びにプログラム
US9564075B2 (en) * 2009-12-30 2017-02-07 Cyweemotion Hk Limited Electronic control apparatus and method for responsively controlling media content displayed on portable electronic device
GB201310364D0 (en) * 2013-06-11 2013-07-24 Sony Comp Entertainment Europe Head-mountable apparatus and systems
KR102083596B1 (ko) * 2013-09-05 2020-03-02 엘지전자 주식회사 디스플레이장치 및 디스플레이장치의 동작방법
CA2939928C (fr) * 2014-02-19 2021-06-22 Evergaze, Inc. Appareil et procede pour optimiser, augmenter ou ameliorer la vision
CN106605196B (zh) * 2014-09-02 2018-11-09 苹果公司 远程相机用户界面
JP6761228B2 (ja) * 2015-06-09 2020-09-23 キヤノン株式会社 表示装置及びその制御方法、プログラム、並びに記憶媒体
US20170214856A1 (en) * 2016-01-22 2017-07-27 Mediatek Inc. Method for controlling motions and actions of an apparatus including an image capture device having a moving device connected thereto using a controlling device
WO2018065986A1 (fr) * 2016-10-06 2018-04-12 Remoria Vr S.R.L. Dispositif de commande de suivi d'orientation et de mouvement
US20180144554A1 (en) * 2016-11-18 2018-05-24 Eyedaptic, LLC Systems for augmented reality visual aids and tools
CN107764262B (zh) * 2017-11-09 2019-10-25 深圳创维新世界科技有限公司 虚拟现实显示设备、系统及姿态校准方法
CN108108015A (zh) * 2017-11-20 2018-06-01 电子科技大学 一种基于手机陀螺仪和动态时间规整的动作手势识别方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110221656A1 (en) * 2010-02-28 2011-09-15 Osterhout Group, Inc. Displayed content vision correction with electrically adjustable lens
US20170336882A1 (en) * 2016-05-17 2017-11-23 Google Inc. Virtual/augmented reality input device

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112733624A (zh) * 2020-12-26 2021-04-30 电子科技大学 室内密集场景的人流密度检测方法、系统存储介质及终端
CN112733624B (zh) * 2020-12-26 2023-02-03 电子科技大学 室内密集场景的人流密度检测方法、系统存储介质及终端

Also Published As

Publication number Publication date
CN112313731A (zh) 2021-02-02
GB2589255A (en) 2021-05-26
CN112424729A (zh) 2021-02-26
GB2588055A (en) 2021-04-14
GB202020076D0 (en) 2021-02-03
WO2019239161A1 (fr) 2019-12-19
GB202020074D0 (en) 2021-02-03

Similar Documents

Publication Publication Date Title
US10983593B2 (en) Wearable glasses and method of displaying image via the wearable glasses
CN109683716B (zh) 基于眼睛跟踪的可见度提高方法和电子装置
JP5093968B2 (ja) カメラ
CN107315512B (zh) 图像处理设备、图像处理方法和程序
US10133407B2 (en) Display apparatus, display system, method for controlling display apparatus, and program
US20060044399A1 (en) Control system for an image capture device
CN111886564B (zh) 信息处理装置、信息处理方法和程序
CN111630477A (zh) 提供增强现实服务的设备及其操作方法
KR102361025B1 (ko) 웨어러블 글래스 및 웨어러블 글래스를 통하여 영상을 디스플레이하는 방법
US20230336865A1 (en) Device, methods, and graphical user interfaces for capturing and displaying media
KR20180004112A (ko) 안경형 단말기 및 이의 제어방법
KR20090064509A (ko) 휴대형 영상 확대장치
WO2019239162A1 (fr) Dispositif portatif pour commander un grossissement numérique sur un dispositif d'affichage portable
US20200073621A1 (en) Systems, Devices, Components and Associated Computer Executable Code For Providing Remote Viewing of a Display Associated with a Computational Device
KR100846355B1 (ko) 두부 장착식 디스플레이 장치의 시력 보조 방법 및 그 두부장착식 디스플레이 장치
CN111061372B (zh) 设备控制方法及相关设备
CN105785749A (zh) 一种显示方法及电子设备
JP2006155172A (ja) 入力装置
US20240320930A1 (en) Devices, methods, and graphical user interfaces for capturing media with a camera application
WO2022123663A1 (fr) Dispositif d'affichage d'image et procédé d'affichage d'image
JP7146944B2 (ja) 表示装置
CN113660477A (zh) 一种vr眼镜及其图像呈现方法
JP2008083074A (ja) カメラ
JP2006195287A (ja) 携帯情報端末

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19732431

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 202020076

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20190617

122 Ep: pct application non-entry in european phase

Ref document number: 19732431

Country of ref document: EP

Kind code of ref document: A1