WO2019160885A1 - Système et procédé de traitement d'un signal vidéo avec une latence réduite - Google Patents

Système et procédé de traitement d'un signal vidéo avec une latence réduite Download PDF

Info

Publication number
WO2019160885A1
WO2019160885A1 PCT/US2019/017742 US2019017742W WO2019160885A1 WO 2019160885 A1 WO2019160885 A1 WO 2019160885A1 US 2019017742 W US2019017742 W US 2019017742W WO 2019160885 A1 WO2019160885 A1 WO 2019160885A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
video signal
array
video
processing
Prior art date
Application number
PCT/US2019/017742
Other languages
English (en)
Inventor
Patrick Murphy
Todd Conard
Douglas J. Glazar
Original Assignee
Freedom Scientific, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US15/895,211 external-priority patent/US10462381B2/en
Application filed by Freedom Scientific, Inc. filed Critical Freedom Scientific, Inc.
Publication of WO2019160885A1 publication Critical patent/WO2019160885A1/fr

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001—Teaching or communicating with blind persons
    • G09B21/008—Teaching or communicating with blind persons using visual presentation of the information for the partially sighted
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders

Definitions

  • This disclosure relates to a video magnifier.
  • the present disclosure also relates to a means of processing a video signal to reduce latency. Also disclosed is a means of video processing whereby shading can be applied to de-emphasize selected portions of the viewing area within the monitor.
  • Video magnifiers are important tools for individuals who are blind or have low vision (i.e.“BLV users”).
  • the construction typically includes an arm mounted camera and an associated monitor.
  • the arm allows the camera to be directly positioned over the object being viewed.
  • the object may be, for example, a prescription pill bottle or a newspaper or book.
  • the arm is usually adjustable to permit the camera to focus upon a particular portion of the object or document.
  • BLV users can then select a desired level of magnification so that the object, and any associated text, can be sufficiently enlarged for viewing and reading. Color contrasting and image rotation can be provided to further ease viewing.
  • An X-Y table can optionally be positioned beneath the camera to allow for precise movements to the object being viewed.
  • a video processor and look up tables are used to process the video signal generated by the camera.
  • the video signal can be processed to make changes to one or more of the following: image scale; image position; color contrast; as well as other image features.
  • Latency refers to the slight delay that results from the video signal being processed. This delay is usually measured in milliseconds and occurs between the image being captured by the camera and its subsequent display upon the monitor.
  • the use of frame buffers generally increases latency.
  • Frame buffers are memory stores that contain a bitmap of an entire frame of data. Frame buffers allow the video frame to be scaled and otherwise allows desired attributes to be applied to the frame.
  • the latency that results from frame buffering is largely unnoticeable in most applications.
  • latency is highly problematic in the context of a video magnifier, as BLV users will notice a lag between the manipulation an object and its subsequent appearance on the video monitor. This lag greatly complicates the manual tasks performed by BLV users.
  • Known video processing systems typically require multiple frame buffers.
  • a typical video processing system there are three frame buffers, one to store the incoming frame, one to store the frame that is being manipulated in memory, and one to store the output frame.
  • the cumulative total of frame buffers times the frame time results in a large delay between an event in front of the camera and when it appears on the display. This delay or video latency causes significant problems for low vision users when they attempt tasks in front of the camera that require eye-hand coordination. Examples include writing during test taking, signing a name to a document, or attempting to push a needle through fabric while sewing. Every frame buffer adds to the cumulative photon to photon latency, which can be measured on a black box system by triggering a bright point source of light in front of the camera system and them measuring the delta in time to when the light is visible on a monitor.
  • Video magnifiers also employ techniques to help focus the user’s vision upon the screen. For this reason, traditional video magnifiers sometimes black out regions of the screen. The blacked out regions tend to highlight the non-blacked out regions. This allows the BLV user to focus upon one area of an object. In the case of text, it allows the BLV user to read one line of text at a time. The underlying text may be scrolled so that sequential lines are presented to the user.
  • the purpose of the mask is to reduce the integral of light entering the user’s eyes.
  • Many low-vision users are sensitive to light and develop eye-strain when using a bright monitor.
  • the masks in a low-vision magnifier are typically placed at the top and bottom of the display output, irrespective of the input scene. This leaves a wide central region where magnified items, typically written words, are visible.
  • users would cut up card board boxes and tape them to their monitors to block off a large percent of the display to help reduce eye-strain from too much light. This method gave rise to the digital horizontal and vertical masks now used in many low-vision magnifier systems. While masks solve the problem of too much light entering the user’s eyes they created a new problem, where the user’s field of view became substantially limited, making the user lose their spatial perspective in the document or scene they were trying to observe during eye-hand coordinated tasks.
  • WO2013/068584 to Eaturnus discloses a ultra-low latency video communication system.
  • This system employs a transmission unit with image acquiring circuitry or image reconstruction circuitry for acquiring or reconstructing an image frame or an image field.
  • a video processing unit is also included for processing at least part of the video data.
  • a communication unit for sending or receiving at least part of the video data.
  • US Pub 2005/0288932 to Kurzweil discloses a system for reducing processing latency in optical character recognition for a portable reading device.
  • the portable reading device may include a computing device with a computer readable medium for storing a computer program to receive an image and select a portion of the image to process. The device then processes the selection of the image with a first process, and when the first process is finished, a second process is initiated. While the second processing, the first process is repeated on another section of the image.
  • US Pub 2013/0329114 to Kim discloses an image magnifier with point control. The method allows for image control using magnification. The method comprises selecting a portion of the image, and then magnifying the selected portion. An action is then performed on the magnified and selected portion of the frame. The change may involve changing a feature of the selected portion.
  • a desktop video magnifier camera includes a pass through signal processor that eliminates the use of frame buffers and that greatly reduces associated latency.
  • the video magnifier of the present disclosure is designed to fulfill this and other shortcomings present with existing video magnifiers.
  • This disclosure relates to a video magnifier with low latency.
  • the disclosed video magnifier has several important advantages. For example, by eliminating the frame buffer, any noticeable delay in signal processing can be avoided.
  • Another advantage is achieved by utilizing a pass through video processor that allows users to view objects in near real time, which is especially important when the objects are being manipulated by BLV users.
  • a further advantage is realized by processing the video signal so that selected portions of the screen are shaded to thereby highlight the non-shaded portions.
  • Yet another advantage is realized by allowing certain portions of an object or text to be highlighted while at the same time allowing the user to perceive the remaining portions of the document.
  • FIG. 1 is a perspective view of the magnifier of the present disclosure.
  • FIG. 2 is a schematic of the video processor used in connection with the present disclosure.
  • FIG. 3 is a front elevational view showing shading for use by blind or low vision users.
  • FIG. 4 illustrates the pixel by pixel signal processing of the present disclosure.
  • FIG. 5 is a flow chart illustrating some of the steps associated with the present disclosure.
  • FIG. 6 is an illustration of an additional embodiment of the present disclosure.
  • This disclosure relates to a video processor for a magnifier camera.
  • the disclosure relates to a video processor that eliminates the use of a frame buffer. This, in turn, reduces the latency otherwise present in the video signal.
  • the disclosed video processor also allows selected portions of the display to be shaded. This highlights the non-shaded portions of the display while at the same time allowing the entire display to be perceived by the user.
  • FIG. 1 illustrates a desktop video magnifier 10 that is constructed accordance with the present disclosure.
  • the particular magnifier depicted is more fully described in co-pending application serial number 14/607,197 filed on January 28, 2015 and entitled “Video Magnifier Camera with Handle.” The contents of this commonly owned application are fully incorporated herein for all purposes.
  • Video magnifier 10 includes a high definition camera 20 that is mounted upon a camera arm 22.
  • Camera arm 22 is flexible. Arm 22 can also swivel about a vertical axis and pivot about one or more horizontal axes to allow the user to properly position camera 20 over the object to be viewed.
  • arm 22 is dimensioned to be received within a trough 24 on the back of the device housing 26.
  • Objects to be viewed can be placed directly beneath camera 20.
  • an x-y table can be included beneath camera 20 to allow objects to be precisely oriented.
  • Magnifier 10 is supported by a base 32.
  • Base 32 supports a video monitor 34 that is coupled to camera 20 via a cable, such as a VGA or HDMI cable.
  • Signals generated by camera 20 are processed via a video processor 36 and displayed on monitor 34 for viewing by the operator.
  • FIG. 2 illustrates processor 36, which is preferably a Field Programmable Gate Array (“FGPA”), and an associated look up table 38 (“LUT”).
  • LUT 38 assigns output values based upon the input signal.
  • the input signal is processed on a pixel-by-pixel basis as opposed to a frame-by-frame basis. Processing the signal on a pixel-by-pixel basis eliminates the need for a frame buffer.
  • FGPA Field Programmable Gate Array
  • processor 36 and LUT 38 can process each individual pixel of the recorded image. This “pass through” processing increases processing times and greatly reduces associated latency.
  • the pixel-by-pixel processing is diagrammatically illustrated in FIG. 4. In order to accomplish this, the camera resolution is matched to the monitor resolution.
  • the frame buffer is eliminated by making the output video synchronous to the input video. As each pixel enters it is manipulated by low- vision processing algorithm(s), in this case by using a dynamically programmable LUT, and then the pixel exits as output video.
  • the delay between input pixel and output pixel is only a single pixel clock, as opposed to the typical three frame delays of other systems which are millions of pixel clocks.
  • Processor 36 and LUT 38 can apply any of a number of different characteristics to the displayed image.
  • the color scheme of the displayed image is altered.
  • images can be displayed in contrasting colors, such as blue/green, red/yellow or black/white. The user may cycle through the various color combinations to find the most effective output.
  • the video signal can be processed to highlight a desired region or regions of the displayed video.
  • This feature of the invention is depicted in FIG. 3.
  • text is being highlighted on the display by shading upper and lower portions (42a and 42b) of the text. This has the desired effect of highlighting the non-shaded portion 42(c).
  • the degree of shading can be varied depending upon the needs of the particular user. It has been found that such shading is beneficial to BLV users suffering from conditions such as retinitis pigmentosa.
  • the shading is preferably not so great so as to completely mask or block out the underling words or images. This prevents the user from losing track of their place within the text. It likewise prevents a user from losing their orientation in an underlying image.
  • the degree of shading and the orientation and positioning of the shading can be selected by the user. For example, the shading can be orientated to highlight a vertical, as opposed to horizontal, portion of the image.
  • the shading In addition to helping focus upon a particular area, the shading also reduces the amount of light entering the user’s eyes. This, in turn, reduces eye strain and lessens the pain associated with some low-vision disabilities.
  • the semi- transparent shades reduce light from the monitor without limiting the viewable area available to the user.
  • FIG. 5 is a flow chart illustrating the steps associated with the present method of video signal processing. As noted, the method is carried out using: a camera; a microprocessor with an associated look-up table; and a monitor. One of the objectives of the method is to process the video signal in a manner that allows the resulting image to be more easily viewed by a blind or low-vision user.
  • the camera is directed toward an object to be viewed.
  • This can be any of a variety of objects the user wishes to examine.
  • the object can be placed underneath the camera or the camera can be pointed to a remote object.
  • the camera then generates a video signal corresponding to the recorded image.
  • This video signal comprises a series of sequential video frames.
  • the image resolution is chosen to match the monitor being employed.
  • Each of the individual video frames in turn, comprises an array of individual pixels or picture elements. The number of pixels is a function of the underlying image resolution. For example, each frame may include a width of 1024 pixels and a height of 768 pixels.
  • the video signal is next delivered to a video processor.
  • the signal is then processed by the microprocessor and in accordance with pre-established instructions from the look-up table.
  • the look-up table specifies portions of the array to be shaded and the degree of shading to be applied.
  • the degree of shading is such that the underlying image is not masked. By not completely masking the underlying image, the user is able to maintain spatial awareness of the object being displayed.
  • the shaded portions of the array function to highlight the non-shaded portions of the array and thereby facilitate viewing by the low-vision user.
  • the shaded portions can be arranged either vertically or horizontally depending upon the preferences of the user.
  • the processed video signal is next delivered to the monitor to be displayed. Because the microprocessor processes the video on a pixel by pixel basis the use of an image store, such as a frame buffer, is eliminated. This, in turn, greatly reduces latency.
  • an image store such as a frame buffer
  • the system utilizes a number of different look up tables (38(a), 38(b), and 38(c)).
  • Each of these look up tables contains instructions for one pre-determ ined visual characteristic to be applied to the underlying image.
  • table 38(a) may contain instructions for shading portions of the array
  • table 38(b) may contain instructions for various color contrasting to be applied to the image and the background
  • table 38(c) may contain instructions for positioning the object within the frame.
  • the present inventors devised a setup to measure latency.
  • the set up consisted of a white LED with a switch, a photo detector, and an oscilloscope. Pressing the switch powered the LED, causing a high intensity photon point source to be injected into the camera sensor array asynchronous to the camera frame rate. Subsequently the photon point source appeared on the output display, with the delta between input event and output event measured on the oscilloscope. Many measurements were taken to find the minimum and maximum latency. Tests with users determined that a cumulative photon to photon latency of less the 100 milliseconds (ms) was required to make low-vision systems usable for tasks requiring eye-hand coordination. The latency for many devices on the market was tested and found unsuitable.
  • the present system overcomes unacceptable latency by eliminating frame buffers in the low-vision enhancement algorithms used with the present video processing systems.
  • the present system also solves the problem of too much light and loss of perspective, by using shades with adjustable opacity that can be enabled and sized both horizontally and vertically. This gives the user the ability to observe the area around the central area of interest without eye-strain, while adjusting the size and opacity of the shades to their individual comfort.
  • the shades and masks of the present system are inserted in real-time into the pixel stream without the use of frame buffers.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Educational Administration (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Health & Medical Sciences (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Studio Devices (AREA)

Abstract

L'invention concerne un processeur vidéo d'une caméra grossissante. En particulier, l'invention concerne un processeur vidéo qui supprime l'utilisation d'un tampon de trame. Ceci, à son tour, réduit la latence présente autrement dans le signal vidéo. Le processeur vidéo selon l'invention permet également d'ombrer des parties sélectionnées de l'affichage. Cela met en évidence les parties non ombrées de l'affichage tout en permettant, en même temps, à l'utilisateur de percevoir l'objet en entier.
PCT/US2019/017742 2018-02-13 2019-02-13 Système et procédé de traitement d'un signal vidéo avec une latence réduite WO2019160885A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/895,211 2018-02-13
US15/895,211 US10462381B2 (en) 2014-04-30 2018-02-13 System and method for processing a video signal with reduced latency

Publications (1)

Publication Number Publication Date
WO2019160885A1 true WO2019160885A1 (fr) 2019-08-22

Family

ID=67618778

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/017742 WO2019160885A1 (fr) 2018-02-13 2019-02-13 Système et procédé de traitement d'un signal vidéo avec une latence réduite

Country Status (1)

Country Link
WO (1) WO2019160885A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070195096A1 (en) * 2006-02-10 2007-08-23 Freedom Scientific, Inc. System-Wide Content-Sensitive Text Stylization and Replacement
US20110299772A1 (en) * 2009-02-13 2011-12-08 Janssen Johannes H M Image processing system for processing a digital image and image processing method of processing a digital image
US20120218404A1 (en) * 2011-02-28 2012-08-30 Nellcor Puritan Bennett Llc Medical Monitor Data Collection System And Method
US20130113903A1 (en) * 2011-11-04 2013-05-09 Research In Motion Limited Image magnification method and apparatus

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070195096A1 (en) * 2006-02-10 2007-08-23 Freedom Scientific, Inc. System-Wide Content-Sensitive Text Stylization and Replacement
US20110299772A1 (en) * 2009-02-13 2011-12-08 Janssen Johannes H M Image processing system for processing a digital image and image processing method of processing a digital image
US20120218404A1 (en) * 2011-02-28 2012-08-30 Nellcor Puritan Bennett Llc Medical Monitor Data Collection System And Method
US20130113903A1 (en) * 2011-11-04 2013-05-09 Research In Motion Limited Image magnification method and apparatus

Similar Documents

Publication Publication Date Title
US10129520B2 (en) Apparatus and method for a dynamic “region of interest” in a display system
US4467349A (en) System and method for composite display
US11551602B2 (en) Non-uniform resolution, large field-of-view headworn display
US9443488B2 (en) Image transforming vision enhancement device
Moshtael et al. High tech aids low vision: a review of image processing for the visually impaired
US20200034575A1 (en) Screen privacy protection method and system for mobile terminal device
JPS6180222A (ja) メガネ合せ方法及びメガネ合せ装置
US10397559B2 (en) Display method, display apparatus, and display system
US20190053607A1 (en) Electronic apparatus and method for providing makeup trial information thereof
KR100345591B1 (ko) 깊이정보처리를위한영상처리시스템
CN116057574A (zh) 色盲辅助技术系统和方法
US11228722B2 (en) System and method for processing a video signal with reduced latency
WO2019160885A1 (fr) Système et procédé de traitement d'un signal vidéo avec une latence réduite
US9891438B2 (en) System and method for processing a video signal with reduced latency
US12058479B2 (en) Full color spectrum blending and digital color filtering for transparent display screens
US20210274127A1 (en) Presentation system and presentation method
DE102015100680B4 (de) Verfahren und Vorrichtungen zur Umgebungsdarstellung
US20120038746A1 (en) Techniques and apparatus for two camera, and two display media for producing 3-D imaging for television broadcast, motion picture, home movie and digital still pictures
JP2001297323A (ja) レンズカラーセレクション方法及び装置
CN112987309A (zh) 基于表面凹凸光栅的融合现实装置
CN115310151A (zh) 一种显示信息保护方法及装置
Koshikizawa et al. Color Appearance Control for Color Vision Deficiency by Projector-Camera System
CN106354247A (zh) 一种头戴式智能眼镜显示控制方法及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19754656

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 30.11.2020)

122 Ep: pct application non-entry in european phase

Ref document number: 19754656

Country of ref document: EP

Kind code of ref document: A1