CN111629238A - Head-mountable imaging device and system for assisting users with reduced vision - Google Patents

Head-mountable imaging device and system for assisting users with reduced vision Download PDF

Info

Publication number
CN111629238A
CN111629238A CN202010118736.4A CN202010118736A CN111629238A CN 111629238 A CN111629238 A CN 111629238A CN 202010118736 A CN202010118736 A CN 202010118736A CN 111629238 A CN111629238 A CN 111629238A
Authority
CN
China
Prior art keywords
head
output
imaging device
video signal
mountable
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010118736.4A
Other languages
Chinese (zh)
Inventor
M·A·阿里
S·希克斯
A·坎贝尔
F·埃雷拉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Renming Hangzhou Intelligent Technology Co ltd
Original Assignee
Renming Hangzhou Intelligent Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Renming Hangzhou Intelligent Technology Co ltd filed Critical Renming Hangzhou Intelligent Technology Co ltd
Publication of CN111629238A publication Critical patent/CN111629238A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/30—Image reproducers
    • H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001—Teaching or communicating with blind persons
    • G09B21/008—Teaching or communicating with blind persons using visual presentation of the information for the partially sighted
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/4104—Peripherals receiving signals from specially adapted client devices
    • H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223—Cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43076—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of the same content streams on multiple devices, e.g. when family members are watching the same movie on different devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
    • H04N21/4363—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network
    • H04N21/43637—Adapting the video or multiplex stream to a specific local network, e.g. a IEEE 1394 or Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622—Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47205—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type

Abstract

A system for assisting a user with reduced vision includes a head-mountable imaging device 30 and a first apparatus 10 separate from the head-mountable imaging device 30. The first apparatus 10 comprises a first input configured to receive media content comprising audio and video and a first output configured to output at least one of an output audio signal and an output video signal to the media presentation apparatus 50, 51. The first apparatus 10 includes a second output configured to transmit the video signal to the head-mountable imaging device 30. The head-mountable imaging apparatus 30 comprises a display device configured to provide a display to the eyes of a user and an input configured to receive a video signal from the first device 10. The head-mountable imaging device 30 includes a processor configured to process an input video signal to provide at least one type of image enhancement for assisting a user with reduced vision.

Description

Head-mountable imaging device and system for assisting users with reduced vision
Technical Field
The present invention relates to a head-mountable imaging device and system for assisting users with reduced vision.
Background
People with various forms of visual impairment, including people with peripheral vision loss and people with central vision loss, have difficulty viewing them because they have difficulty focusing on distant objects. This is associated with many everyday activities, including watching television, as the television is typically placed at a distance from the viewer's location. This can be an obstacle for visually impaired people with or without peripheral vision, who often cannot focus on distant objects. Therefore, these people often have to sit very close to the television in order to watch it with acceptable quality. Many visually impaired people may be even more inconvenienced by having to rely solely on sound guides such as Audio Descriptions (AD).
It is an object of the present invention to address at least one of the disadvantages associated with the prior art.
Disclosure of Invention
A system for assisting a user with reduced vision is provided, comprising:
head-mountable imaging apparatus and
a first apparatus separate from the head-mountable imaging device, the first apparatus comprising:
a first input configured to receive media content comprising audio and video;
a first output configured to output at least one of an output audio signal and an output video signal to a media presentation device;
a second output configured to transmit the video signal to a head-mountable imaging device; and
the head-mountable imaging apparatus includes:
a display device configured to provide a display to an eye of a user;
an input configured to receive a video signal from a first apparatus;
a processor configured to process the input video signal to provide at least one type of image enhancement for assisting a user with reduced vision.
Optionally, the second output comprises a wireless transmitter configured to transmit the video signal to a head-mountable imaging apparatus, and the head-mountable imaging apparatus comprises a wireless receiver configured to receive the video signal from the first device. Alternatively, the second output of the first device may be connected to the input of the head-mountable imaging apparatus by a cable.
Optionally, the head-mountable imaging apparatus includes:
a camera configured to provide a video signal representing a scene in front of an imaging device;
and wherein the head-mountable imaging apparatus is configured to selectively operate in a first mode in which a video signal from the camera is processed and output to the display device, and a second mode in which a video signal received from the first device is processed and output to the display device.
Optionally, the first apparatus comprises a processor configured to introduce a delay between an output audio signal output to the media presentation apparatus and a video signal transmitted to the head-mountable imaging device.
Optionally, the value of the delay is preconfigured.
Optionally, the value of the delay is user configurable.
Optionally, the first apparatus is configured to selectively operate in:
a first mode in which the wireless transmitter is configured to transmit a video signal to the head-mountable imaging device and the processor is configured to introduce a delay between an output audio signal output to the media presentation apparatus and the video signal transmitted to the head-mountable imaging device; and
a second mode in which the wireless transmitter is not configured to transmit the video signal to the head-mountable imaging device and the processor is not configured to introduce a delay between the output audio signal output to the media presentation apparatus and the video signal transmitted to the head-mountable imaging device.
Optionally, the image enhancement comprises at least one of:
edge detection and presentation of detected edges;
contrast enhancement;
enlargement of a selected area of the entire image.
Optionally, the first device is a set-top box, the first input is configured to receive at least one of: terrestrial television signals, satellite television signals, and cable television signals, and the first device includes a tuner.
Optionally, the first device is configured to receive an Internet Protocol Television (IPTV) signal.
Optionally, the set-top box comprises a media storage device and the first device is capable of recording and playing back media content.
Optionally, the first device is configured to connect to a host device, the first input comprising a wireless interface configured to receive the media content as a data stream from a server.
Optionally, the first device is configured to connect to an HDMI port of the host device.
There is provided a device for assisting a user with reduced vision, comprising:
a first apparatus separate from the head-mountable imaging device, the first apparatus comprising:
a first input configured to receive media content comprising audio and video;
a first output configured to output at least one of an output audio signal and an output video signal to a media presentation device;
a wireless transmitter configured to transmit a video signal to a head-mountable imaging device; and
a processor configured to introduce a delay between an output audio signal output to the media presentation device and a video signal transmitted to the head-mountable imaging apparatus.
Optionally, the value of the delay is preconfigured.
Optionally, the value of the delay is user configurable.
Optionally, the first apparatus is configured to selectively operate in:
a first mode in which the wireless transmitter is configured to transmit a video signal to the head-mountable imaging device and the processor is configured to introduce a delay between an output audio signal output to the media presentation apparatus and the video signal transmitted to the head-mountable imaging device; and
a second mode in which the wireless transmitter is not configured to transmit the video signal to the head-mountable imaging device and the processor is not configured to introduce a delay between the output audio signal output to the media presentation apparatus and the video signal transmitted to the head-mountable imaging device.
Optionally, the first apparatus comprises a processor configured to process the input video signal to provide at least one type of image enhancement for assisting a user with reduced vision. The image enhancement performed by the processor at the first apparatus may be used in conjunction with or in place of any image enhancement performed by the head-mountable imaging device. An advantage of performing image enhancement at the first device may be an improved processing performance. An advantage of performing image enhancement at the first apparatus may be to reduce energy consumption at the head-mountable imaging device, since less processing is required. This may extend battery life. Optionally, only the first apparatus has a processor configured to process the input video signal to provide at least one type of image enhancement to assist the visually impaired user, and there is no processing capability at the head-mountable imaging device.
An advantage of at least one example is to allow a visually impaired user to view media content, such as a live television signal or other media content, with improved quality.
An advantage of at least one example is to allow a visually impaired user to view media content and listen to audio of the media content via a head-mountable imaging device, wherein synchronization between the image and the audio is improved.
The functions described herein may be implemented in hardware, software executed by a processing device, or a combination of hardware and software. The processing device may include a computer, processor, state machine, logic array, or any other suitable processing device. The processing device may be a general-purpose processor that executes software to cause the general-purpose processor to perform desired tasks or the processing device may be dedicated to performing desired functions. Another aspect of the invention provides machine-readable instructions (software) which, when executed by a processor, perform any of the methods described. The machine-readable instructions may be stored on an electronic storage device, hard disk, optical disk, or other machine-readable storage medium. The machine-readable medium may be a non-transitory machine-readable medium. The term "non-transitory machine-readable medium" includes all machine-readable media except transitory propagating signals. The machine-readable instructions may be downloaded to the storage medium via a network connection.
It is envisaged within the scope of the present application that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, claims and/or in the following description and drawings, in particular the various features thereof, may be employed individually or in any combination. For example, features described in connection with one embodiment are applicable to all embodiments unless the features are incompatible.
For the avoidance of doubt, it is to be understood that features described in relation to one aspect of the invention may be included in any other aspect of the invention, either alone or in appropriate combination with one or more other features.
Drawings
One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which:
figures 1 and 2 show a first example of a system;
FIG. 3 shows a set-top box for use in the system;
FIG. 4 illustrates a head-mountable imaging device for use in the system;
FIG. 5 illustrates a handle of a head-mountable imaging device;
6A-6D illustrate examples of image enhancement;
fig. 7 shows an example of an apparatus for use in place of a set-top box.
Detailed Description
Fig. 1-4 illustrate examples of systems for assisting users with reduced vision. The system includes a set-top box (STB)10, a head-mountable imaging device 30, and a media presentation apparatus in the form of a display (television 50) and audio speakers 51.
The STB10 includes an input port 11 for receiving television signals. The television signal may be a terrestrial television signal, a satellite television signal, or a cable television signal. Television signals are typically modulated in the Radio Frequency (RF) band. The STB10 includes a television tuner 12 for downconverting a selected television channel from RF to baseband. The demodulator demodulates the television signal selected by the tuner. The demodulator may be provided as part of the tuner 12 or may be provided separately. The decoder decodes the audio and video data. The demodulator outputs a digital signal including audio and video in digital form. A range of modulation and coding schemes are used. For example, digital video broadcasting-terrestrial (DVB-T) uses Orthogonal Frequency Division Multiplexing (OFDM) and Moving Picture Experts Group (MPEG) MPEG-2 or MPEG-4 encoding. Some television formats (e.g., DVB-T) also include data. The data may be used to carry caption information for a television channel. The audio, video (and optionally data) together constitute media content, such as a television program, a movie, or some other media content. The STB10 may be configured to receive IPTV signals without, or in addition to, other types of television signals described above.
The STB10 includes audio and video outputs 18 for connection to an external media presentation device. The audio and video output may take the form of digital output (e.g., High Definition Multimedia Interface (HDMI), universal serial bus type C (USB-C)) and/or analog output (e.g., phono/RCA ports and headphone ports for audio output). The external media presentation device may be a display, such as a television. The external media presentation devices may be an audio amplifier and speakers.
The STB10 may include a port 13 for connecting to a media storage device. The port 13 may be a port to receive a micro SD card, a Universal Serial Bus (USB) port to connect to an external disk drive or another video stream input device. The STB may stream content from one of the storage devices using port 13.
The STB10 includes a wireless transmitter 19 for transmitting to the head-mountable imaging device 30. The wireless transmitter 19 may be BluetoothTMA transmitter, or a transmitter configured to use another short-range wireless protocol. The STB10 uses the wireless transmitter 19 to transmit video data to the head-mountable imaging device 30.
STB10 may include a wireless transmitter (e.g., wireless transmitter 19 or a separate wireless transmitter) for transmitting audio data to a wireless headset or wireless earphone.
The STB10 may also include media storage 21. Media storage 21 may be used to store media content for later playback or to provide a paused live television feature. The STB10 may provide the functionality of a conventional Digital Video Recorder (DVR).
Referring to fig. 1, the system may include a desktop device/computer with a power connection; smart glasses with or without optical element image processing functions and wireless modules; and smart glasses controller or remote control (add pictures). The proposed system may also include a television set with a wired connection or a wireless module to receive television signals from the set-top box.
Fig. 4 shows an example of the head-mountable imaging apparatus 30. The imaging device 30 may take the form of a pair of head-mountable glasses, but it may take the form of a headset. The imaging device 30 shown in fig. 2 has a frame that is worn in a manner similar to a conventional pair of eyeglasses. The head-mountable imaging device 30 comprises a camera 31, which camera 31 is configured to provide a video signal 32 representing a scene in front of the imaging device. The imaging device 30 may have a single camera 31, or multiple cameras, for example one for each eye. The imaging device 30 may include a left eye display 38 and a right eye display 39. The processor 35 receives the video signal 31 and outputs the processed video signal to the displays 38, 39. The processor 40 is configured to perform image processing to assist users with impaired vision. Examples of image processing are shown in fig. 6A-6D. The imaging device 30 may include a transparent screen or lens that allows the user to view the displayed image in combination with the surrounding environment through the screen/lens, or may include an opaque screen that only allows the user to view the displayed image. The head-mountable imaging device 30 may be referred to as "smart glasses".
The imaging device 30 can be used in two possible modes. In the first mode, imaging device 30 provides image processing enhancements to assist the user in viewing the surrounding environment. The video signal 32 from the camera 31 is processed to assist the user and output to the displays 38, 39. In the second mode, the imaging device 30 provides image processing enhancements to assist the user with the media content received from the first apparatus 10. The video signal 34 received from the first device 10 is processed and output to the displays 38, 39.
The head-mountable imaging device 30 may include one or more controls for allowing a user to control image processing. For example, the control may allow the user to control parameters of the image processing, such as controlling the number of edges displayed or selecting a portion of the image to zoom in. One or more controls may be located on a portion 40 of the device that is worn on the user's head, such as the arms of the glasses. One or more controls 43 may be located on the handle 42. The handle 42 may be connected to the head-mounted portion 40 by a cable 41 or by a wireless interface.
In some examples, the processor 35 of the imaging device 30 is located within a portion 40 of the device that is worn on the user's head. In other examples, the processor 35 may be located in a box that is physically separate from the portion 40 of the device that is worn on the user's head. Handle 42 may house processor 35. A battery may also be provided within the handle 42 to power the device 30.
The system enables visually impaired users to stream live television signals (or other media content) from STB10 to their glasses 30 and enhance the live television signals using processor 35 on glasses 30. For example, the user may select an object or region of interest to zoom in on the object of interest in real-time. Processor 35 may apply image processing filters that have proven beneficial for vision disorders.
In one aspect, television signals may be streamed in real-time to multiple types of display systems including both smart glasses and television. In this regard, rather than using the camera 31 on the glasses 30, a visually impaired person may view an image from a closer distance (e.g., in High Definition (HD) quality), while other viewers may view the same image on the television screen 50.
In one aspect, visually impaired people can view objects from closer distances with HD quality and apply image processing filters such as different levels of contrast, while their family can view the same image on a television screen without the need for image processing filters.
One problem with the system shown in fig. 2 is that the image displayed on the head-mountable imaging device 30 may lag behind the audio transmitted via the television 50 or speakers 51. This lag may be due to the difference of the two processing paths:
path 1 — STB10 to head-mountable device 30 and includes delays due to wireless transmission (delay at transmitter 19, delay in transmission, delay at receiver 33) and delays due to processing at processor 35 before display to the user;
path 2 to the STB10 or to the television 50 or to the speaker 51.
The system may introduce a delay in the path of the audio signal (and video signal) output to port 20. This may help synchronize the image displayed on the head-mountable imaging device 30 with the audio transmitted via the television 50 or the speaker 51. Other viewers of the television 50 will watch the image and audio simultaneously and will not be aware of the delay introduced in the path to the television 50. The value of the delay may be based on the length of time it takes to transmit to the glasses 30 generally. In this way both parties can listen to the audio from the television speakers and view the images simultaneously with their preferred displays. There are a number of ways to implement the delay. One possibility is for the processor 15 to read out: (i) audio data (and video data) to output port 20 and (ii) video data to wireless transmitter 19. The processor 15 may be configured to read out the audio data (and video data) to the output port 20 at a later time than the video data to the wireless transmitter 19.
The value of the delay may be pre-configured or may be user-configurable. For example, the user may configure the delay by using controls on the head-mountable imaging device 30 (e.g., controls on the handle 42). Alternatively, the user may configure the delay by using controls on the STB10 or a user interface of the STB 10.
The handle 42 may include one or more sensors that may detect movement of the handle. One or more accelerometers may be provided within the handle to measure motion along different axes, such as an x-axis accelerometer, a y-axis accelerometer, and a z-axis accelerometer. Fig. 5 shows two possible forms of motion detection for the handle 42: movement about an axis of rotation (in this example, the axis of rotation is the longitudinal axis of the handle 42); along an axis (which in this example is along or parallel to the longitudinal axis of the handle 42). In one example, the value of the delay may be adjusted by rotating the handle, e.g., clockwise to increase the delay and counterclockwise to decrease the delay. In one example, the value of the delay may be adjusted by moving the handle along the axis, e.g., forward to increase the delay and backward to decrease the delay.
The STB10 is configured to selectively operate in:
a first mode in which the wireless transmitter 19 is configured to transmit a video signal to the head-mountable imaging device 30 and the processor 15 is configured to introduce a delay between the output audio signal output to the media presentation apparatus 50 and the video signal transmitted to the head-mountable imaging device 30; and
a second mode in which the wireless transmitter 19 is not configured to transmit video signals to the head-mountable imaging device 30 and the processor 15 is not configured to introduce a delay between the output audio signals output to the media presentation apparatus 50 and the video signals transmitted to the head-mountable imaging device 30.
The processor 15 comprises a mode selector function 16 to allow selection of the first mode and the second mode. The mode may be selected via a control or user interface of STB 10. The mode may be selected by data transmitted from the head-mountable imaging apparatus 30.
Fig. 6A-6D illustrate some examples of image processing for assisting a user with reduced vision. Fig. 6A shows an example of changing the brightness of an image. This can be applied to color or grayscale images. Fig. 6B shows an example of edge detection and rendering the detected edge as a white edge on a black background. Fig. 6C shows an example of edge detection and rendering the detected edge as a white edge overlaid on a color or grayscale image. Fig. 6D illustrates an example of enhanced contrast between image features. This can be done with a global threshold applied to the entire screen to render a black and white high contrast image, or with multiple region thresholds to compensate for illumination variations across the screen.
Other possible image processing enhancements may include an algorithm that detects large regions of similar hue (e.g., independent of brightness) and then renders these regions as a high brightness palette of the same color. Processor 35 may perform other enhancements or image processing. Other processing functions include one or more of: zooming in or out; display of a high resolution static image; the display of picture-in-picture; text identification and recognition. The type of enhancement or image processing performed by processor 35 may depend on the user's visual impairment.
Fig. 7 shows an example of a device 110 that may be used in place of the STB 10. The device 110 is intended to be an accessory (e.g., a dongle or small device) that can be plugged directly into an input port of a host device (e.g., an HDMI port) or connected to an input port of a host device via a cable and connector. The host device 150 may be a display or television 50, a computer (e.g., a laptop or tablet computer), or some other media presentation device. The device 110 has a wireless receiver 111 to receive a media content stream from a server. The wireless receiver may use WiFi and may stream content from a local or remote server, such as an internet server, via a Wireless Local Area Network (WLAN). The apparatus 110 comprises at least one audio/video output 120 for connection to a host device 150, an external media presentation device. The audio/video output 118 may be an HDMI port, a USB-C port, or any suitable port. The device 110 may be powered through a power connection 121 (e.g., USB) to the host device 150 or via a separate power source.
The apparatus 110 includes a wireless transmitter 119 for transmitting to the head-mountable imaging device 30. The wireless transmitter 119 may be BluetoothTMA transmitter, or a transmitter configured to use another short-range wireless protocol. The apparatus 110 transmits video data to the head-mountable imaging device 30 using the wireless transmitter 119.
The apparatus 110 of fig. 7 may have an input port 125 in addition to the wireless receiver 111. Alternatively, the apparatus 110 of fig. 7 may have the input port 125 without the wireless receiver 111. The input port 125 may be an HDMI port, a USB-C port, or any other suitable type of port for receiving media content. The user may connect a source of media content to the input port 125 via a suitable conductor. The source of the media content may be a smartphone, tablet, computer, or any other device that can output the media content to the device 110.
The device 110 may have similar functionality as described above with respect to the STB 10. It may introduce a delay in the path of the audio signal (and video signal) output to port 120. This may help synchronize the image displayed on the head-mountable imaging apparatus 30 with the audio transmitted via the host device 150. The options for selecting the delay are the same as described above.
Alternatively, the STB10 or device 110 may process the input video signal to provide at least one type of image enhancement 23, 123 for assisting users with impaired vision. The image enhancement 23, 123 performed by the STB or the processor at the apparatus 110 may be used in conjunction with any image enhancement performed by the head-mountable imaging device 30, or in place of any image enhancement performed by the head-mountable imaging device 30. This may help reduce power requirements at the head-mountable imaging device 30 and extend battery life.
In fig. 3, 4, 6, and 7, the processors 15, 35, 115 may be implemented as one or more processors, which may be microprocessors, controllers, or any other suitable type of processor for executing instructions to control the operation of the device. Any computer readable medium, such as memory 22, 41, 122, may be used to provide the processor executable instructions. The processor-executable instructions may include instructions for implementing the functionality of the described methods. The memory is of any suitable type, for example Read Only Memory (ROM), Random Access Memory (RAM), any type of storage device such as magnetic or optical storage.
Throughout the description and claims of this specification, the words "comprise" and "contain" and variations of the words, for example "consisting of … …" and "comprising", mean "including but not limited to", and are not intended to (and do not) exclude other moieties, additives, components, integers or steps.
Throughout the description and claims of this specification, the singular encompasses the plural unless the context otherwise requires. In particular, where the indefinite article is used, the specification is to be understood as contemplating plurality as well as singularity, unless the context requires otherwise.
Features, integers, characteristics, compounds, chemical moieties or groups described in conjunction with a particular aspect, embodiment or example of the invention are to be understood to be applicable to any other aspect, embodiment or example described herein unless incompatible therewith.

Claims (17)

1. A system for assisting a user with reduced vision, comprising:
head-mountable imaging apparatus and
a first apparatus separate from the head-mountable imaging device, the first apparatus comprising:
a first input configured to receive media content comprising audio and video;
a first output configured to output at least one of an output audio signal and an output video signal to a media presentation device;
a wireless transmitter configured to transmit a video signal to the head-mountable imaging device; and
the head-mountable imaging apparatus includes:
a display device configured to provide a display to an eye of a user;
a wireless receiver configured to receive a video signal from the first device;
a processor configured to process the input video signal to provide at least one type of image enhancement for assisting a user with reduced vision.
2. The system of claim 1, wherein the head-mountable imaging device comprises:
a camera configured to provide a video signal representing a scene in front of an imaging device;
and wherein the head-mountable imaging apparatus is configured to selectively operate in a first mode in which a video signal from the camera is processed and output to the display device,
in the second mode, a video signal received from the first device is processed and output to the display device.
3. The system of claim 1 or 2, wherein the first device comprises a processor configured to introduce a delay between an output audio signal output to the media presentation device and a video signal transmitted to the head-mountable imaging apparatus.
4. The system of claim 3, wherein the value of the delay is preconfigured.
5. The system of any preceding claim, wherein the value of the delay is user configurable.
6. The system of any of claims 3 to 5, wherein the first device is configured to selectively operate to:
a first mode in which the wireless transmitter is configured to transmit a video signal to the head-mountable imaging device and the processor is configured to introduce a delay between an output audio signal output to the media presentation apparatus and the video signal transmitted to the head-mountable imaging device; and
a second mode in which the wireless transmitter is not configured to transmit video signals to the head-mountable imaging device and the processor is not configured to introduce a delay between output audio signals output to the media presentation apparatus and video signals transmitted to the head-mountable imaging device.
7. The system of any preceding claim, wherein the image enhancement comprises at least one of:
edge detection and presentation of detected edges;
contrast enhancement;
enlargement of a selected area of the entire image;
text identification and recognition.
8. The system of any preceding claim, wherein the first device is a set top box, the first input being configured to receive at least one of: a terrestrial television signal, a satellite television signal, and a cable television signal, and the first device includes a tuner.
9. The system of any preceding claim, wherein the set top box comprises a media storage device and the first device is capable of recording and playing back media content.
10. A system according to any of claims 1 to 7, wherein the first device is configured to receive IPTV signals.
11. The system of any one of claims 1 to 7, wherein the first device is configured to connect to a host device.
12. The system of claim 11, wherein the first input comprises a wireless interface configured to receive media content as a data stream from a server.
13. The system of claim 11 or 12, wherein the first device is configured to connect to an HDMI port of the host device.
14. An apparatus for assisting a user with reduced vision, comprising:
a first apparatus separate from a head-mountable imaging device, the first apparatus comprising:
a first input configured to receive media content comprising audio and video;
a first output configured to output at least one of an output audio signal and an output video signal to a media presentation device;
a wireless transmitter configured to transmit a video signal to the head-mountable imaging device; and
a processor configured to introduce a delay between an output audio signal output to the media presentation device and a video signal transmitted to the head-mountable imaging apparatus.
15. The system of claim 14, wherein a value of the delay is preconfigured.
16. The system of claim 14 or 15, wherein the value of the delay is user configurable.
17. The system of any of claims 14 to 16, wherein the first device is configured to selectively operate to:
a first mode in which the wireless transmitter is configured to transmit a video signal to the head-mountable imaging device and the processor is configured to introduce a delay between an output audio signal output to the media presentation apparatus and the video signal transmitted to the head-mountable imaging device; and
a second mode in which the wireless transmitter is not configured to transmit video signals to the head-mountable imaging device and the processor is not configured to introduce a delay between output audio signals output to the media presentation apparatus and video signals transmitted to the head-mountable imaging device.
CN202010118736.4A 2019-02-27 2020-02-26 Head-mountable imaging device and system for assisting users with reduced vision Pending CN111629238A (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
GBGB1902664.0A GB201902664D0 (en) 2019-02-27 2019-02-27 Head mountable imaging apparatus and system for assisting a user with reduced vision
GB1902664.0 2019-02-27
GB2002406.3 2020-02-20
GB2002406.3A GB2583822B (en) 2019-02-27 2020-02-20 Head mountable imaging apparatus and system for assisting a user with reduced vision

Publications (1)

Publication Number Publication Date
CN111629238A true CN111629238A (en) 2020-09-04

Family

ID=65999027

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010118736.4A Pending CN111629238A (en) 2019-02-27 2020-02-26 Head-mountable imaging device and system for assisting users with reduced vision

Country Status (2)

Country Link
CN (1) CN111629238A (en)
GB (2) GB201902664D0 (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102611903A (en) * 2011-01-17 2012-07-25 索尼公司 3D motion picture processing device
CN102763061A (en) * 2009-09-30 2012-10-31 联合视频制品公司 Systems and methods for navigating a three-dimensional media guidance application
US20120274750A1 (en) * 2011-04-26 2012-11-01 Echostar Technologies L.L.C. Apparatus, systems and methods for shared viewing experience using head mounted displays
US20140112506A1 (en) * 2012-10-19 2014-04-24 Sony Europe Limited Directional sound apparatus, method graphical user interface and software
CN105490892A (en) * 2014-10-01 2016-04-13 索尼公司 Presentation of enlarged content on companion display device
US9338391B1 (en) * 2014-11-06 2016-05-10 Echostar Technologies L.L.C. Apparatus, systems and methods for synchronization of multiple headsets
US20170246070A1 (en) * 2016-02-29 2017-08-31 Osterhout Group, Inc. Radar imagery in head-worn computing
US20180125716A1 (en) * 2016-11-10 2018-05-10 Samsung Electronics Co., Ltd. Visual aid display device and method of operating the same

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3001066C (en) * 2015-10-05 2021-02-23 Esight Corp. Methods for near-to-eye displays exploiting optical focus and depth information extraction

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102763061A (en) * 2009-09-30 2012-10-31 联合视频制品公司 Systems and methods for navigating a three-dimensional media guidance application
CN102611903A (en) * 2011-01-17 2012-07-25 索尼公司 3D motion picture processing device
US20120274750A1 (en) * 2011-04-26 2012-11-01 Echostar Technologies L.L.C. Apparatus, systems and methods for shared viewing experience using head mounted displays
US20140112506A1 (en) * 2012-10-19 2014-04-24 Sony Europe Limited Directional sound apparatus, method graphical user interface and software
CN105490892A (en) * 2014-10-01 2016-04-13 索尼公司 Presentation of enlarged content on companion display device
US9338391B1 (en) * 2014-11-06 2016-05-10 Echostar Technologies L.L.C. Apparatus, systems and methods for synchronization of multiple headsets
US20170246070A1 (en) * 2016-02-29 2017-08-31 Osterhout Group, Inc. Radar imagery in head-worn computing
US20180125716A1 (en) * 2016-11-10 2018-05-10 Samsung Electronics Co., Ltd. Visual aid display device and method of operating the same

Also Published As

Publication number Publication date
GB2583822B (en) 2022-05-25
GB201902664D0 (en) 2019-04-10
GB2583822A (en) 2020-11-11
GB202002406D0 (en) 2020-04-08

Similar Documents

Publication Publication Date Title
US8994795B2 (en) Method for adjusting 3D image quality, 3D display apparatus, 3D glasses, and system for providing 3D image
EP2577981B1 (en) Method and apparaus for making intelligent use of active space in frame packing format
US20130038611A1 (en) Image conversion device
JP4937369B2 (en) Electronic device, video output system, and video output method
US20140362198A1 (en) Stereoscopic Video Processor, Stereoscopic Video Processing Method and Stereoscopic Video Processing Program
JP4693918B2 (en) Image quality adjusting apparatus and image quality adjusting method
US20120050509A1 (en) Stereoscopic glasses, stereoscopic video display device, and stereoscopic video display system
WO2016063617A1 (en) Image generation device, image extraction device, image generation method, and image extraction method
CN102202206B (en) Communication equipment
JP2019083504A (en) Hardware system for inputting stereoscopic image in flat panel
US8537207B2 (en) Video-audio playing system relating to 2-view application and method thereof
CN111629238A (en) Head-mountable imaging device and system for assisting users with reduced vision
JP2012134655A (en) Image processing device, image processing method, and image display device
US20120281073A1 (en) Customization of 3DTV User Interface Position
US20120268559A1 (en) Electronic apparatus and display control method
US20120154383A1 (en) Image processing apparatus and image processing method
US20120162398A1 (en) Stereoscopic Video Processor and Stereoscopic Video Processing Method
US20120154382A1 (en) Image processing apparatus and image processing method
US20130047186A1 (en) Method to Enable Proper Representation of Scaled 3D Video
US20120249754A1 (en) Electronic apparatus, display control method for video data, and program
US20150189257A1 (en) Electronic device and method for controlling the same
JP5377731B2 (en) Video processing apparatus and video processing method
US20140028795A1 (en) Image Processing Method and Image Displaying system
JP2012175339A (en) Three-dimensional video signal processing apparatus and processing method
US20120019630A1 (en) Image output apparatus and image output method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination