US20170069228A1 - Vision Assistive Devices and User Interfaces - Google Patents
Vision Assistive Devices and User Interfaces Download PDFInfo
- Publication number
- US20170069228A1 US20170069228A1 US15/268,927 US201615268927A US2017069228A1 US 20170069228 A1 US20170069228 A1 US 20170069228A1 US 201615268927 A US201615268927 A US 201615268927A US 2017069228 A1 US2017069228 A1 US 2017069228A1
- Authority
- US
- United States
- Prior art keywords
- housing
- assistive device
- vision assistive
- control system
- vision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/001—Teaching or communicating with blind persons
- G09B21/008—Teaching or communicating with blind persons using visual presentation of the information for the partially sighted
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B17/00—Details of cameras or camera bodies; Accessories therefor
- G03B17/02—Bodies
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/0035—User-machine interface; Control console
- H04N1/00496—Constructional details of the interface or console not otherwise provided for, e.g. rotating or tilting means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00519—Constructional details not otherwise provided for, e.g. housings, covers
- H04N1/00559—Mounting or support of components or elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/00519—Constructional details not otherwise provided for, e.g. housings, covers
- H04N1/00562—Supporting the apparatus as a whole, e.g. stands
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/04—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
- H04N1/19—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays
- H04N1/195—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays the array comprising a two-dimensional array or a combination of two-dimensional arrays
- H04N1/19594—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays the array comprising a two-dimensional array or a combination of two-dimensional arrays using a television camera or a still video camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/51—Housings
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
-
- H04N5/2252—
-
- H04N5/247—
Definitions
- the present disclosure relates to vision assistive devices, and in particular, to free-standing vision assistive devices with one or more imaging units.
- vision assistance devices There are a number of vision assistance devices on the market to assist those whose ability to read is impaired by visual difficulties.
- one problem with the current devices is that the current devices require complex set up before they can be used.
- operating current vision assistance devices can be problematic for a visually impaired person due to the heavy reliance of the current vision assistance devices on digital controls, such as a keypad, keyboard, a mouse, or a touch screen. Accordingly, there is still a need for a vision assistive device that is easy to set up and operate.
- a vision assistive device that includes a housing having a base for positioning the housing on a surface; one or more imaging units disposed along a top portion of the housing, each imaging unit being angled downward so a central axis of the imaging unit forms an acute angle with the surface so the imaging unit captures a target image from a target object; and a control system accommodated within the housing, the control system being connected to the one or more imaging units to process the target image captured by each imaging unit and to output the target image to a user.
- a vision assistive device that includes a housing having a base for positioning the housing on a surface; one or more imaging units configured to capture a target image from a target object positioned substantially perpendicular to the housing; a screen integrated with the housing; a control system accommodated within the housing, the control system being connected to the imaging units to process data corresponding to the image captured by the imaging units and to output the target image to a user on the screen.
- a method for providing vision assistance includes disposing a housing substantially perpendicular to a surface, the housing including one or more imaging units and a control system; capturing with the one or more imaging units a target image from a target object placed on the surface adjacent to the housing; and processing the captured target image and outputting the target image on a screen integrated with the housing.
- FIG. 1 is a prospective view on an embodiment of a vision assistive device of the present disclosure.
- FIG. 2 illustrates an embodiment of a vision assistive device of the present disclosure placed on a surface for capturing an image from a target.
- FIG. 3 is a perspective view of the front of the magnifying apparatus.
- FIG. 3A , FIG. 3B , and FIG. 3C illustrate an embodiment of a vision assistive device of the present disclosure having a slidable neck.
- FIG. 4 , FIG. 5 , and FIG. 6 illustrate an embodiment mechanism for operating a slidable neck.
- FIG. 7 illustrates various positions of imaging units over a target by a vision assistive device of the present disclosure.
- FIG. 8 illustrates an embodiment of a vision assistive device of the present disclosure with integrated imaging units.
- FIG. 9 and FIG. 10 illustrate various locations of imaging units relative to a target.
- FIG. 11 illustrates an embodiment of a control system for controlling functionalities of a vision assistive device of the present disclosure.
- FIG. 12 is a front view on an embodiment of a vision assistive device of the present disclosure showing various control features of the vision assistive device;
- the present disclosure provides vision assistive devices, systems and methods of use of vision assistive devices and systems.
- a vision assistive device 100 of the present disclosure includes a housing 42 .
- the housing 42 may be made of separate parts integrated into a single unit.
- the housing 42 may include a base region 62 and a top region 64 connected by a front wall 66 , back wall 68 , and two side walls 70 , 72 .
- the walls of the housing 42 may be continuous and, together with the base region 62 and the top region 64 , define an inner volume for accommodating therein electrical components (not shown) for carrying out functionalities of the vision assistive device 100 , as is described below.
- the housing 42 can be free standing.
- the base region 62 of the housing 42 may be designed to enable the vision assistive device 100 to stand unsupported or unassisted on a surface 86 in operation. In this manner, the vision assistive device 100 can simply be placed on a surface 86 , without any further manipulations.
- the base region 62 of the housing 42 may be substantially planar to enable the vision assistive device 100 to stand unassisted on the surface 86 , substantially perpendicular to the surface.
- a target 88 with a target image (text, indicia, graphics or other images) that the user needs the vision assistance device 100 to capture from the target 88 can be placed on the surface 86 adjacent to the vision assistive device 100 for capturing the images on the target object of paper by the vision assistive device 100 , and subsequently communicating the images to the user.
- the target 88 may be placed such that one edge of the target is along the housing 42 .
- the surface of the target 88 on which the image is located is preferably substantially planar, however, other shapes may also be accommodated.
- the housing 42 may include a foldable or telescopic handle 43 to facilitate carrying the vision assistive device 100 .
- the housing 42 may include a plurality of ports 52 for connecting the electrical components housed inside the housing 42 to external devices and power sources.
- the housing 42 may include one or more USB, VGA, HDMI, Ethernet, and other ports for connecting the vision assistive device 100 to external device, including, but not limited to, external data storage, phones, computers, speakers, televisions, monitors, printers and similar devices.
- the housing 42 may further include one or more slots for insertion of a smart card or memory card.
- the housing 42 can include wireless interfaces, such as WiFi or Bluetooth.
- the housing 42 may also support a plurality of control buttons to allow the user to control one or more functions of the vision assistive device 100 , which control buttons and functions are described in details below.
- the vision assistive device 100 may include one or more imaging units 44 disposed in the top region 64 of the housing 42 .
- the number of imaging units may vary depending on, for example, the type of the imaging units employed or the size or shape of the target 88 .
- the vision assistive device 100 may include a single imaging unit.
- the vision assistive device 100 may include two imaging units.
- the vision assistive device 100 may include more than two imaging units.
- the imaging units 44 may be generally down-facing (but not necessarily pointed straight down) to be able to capture the images from the target 88 placed substantially perpendicular to the vision assistive device 100 , which is placed, horizontally on the surface 86 on which the vision assistive device 100 is placed. In some embodiments, the imaging units 44 may be oriented relative to the target 88 so the axis of the viewing lens of the imaging units 44 is substantially perpendicular to the surface of the target 88 . In other embodiments, the imaging units 44 may be oriented relative to the target 88 so the axis of the viewing lens of the imaging units 44 is at an acute angle to the surface 86 .
- the imaging unit 44 is oriented with the lens axis at an acute angle to the surface 86 and the lens directing on the center or near the center of the target 88 placed on the surface 86 .
- the imaging units 44 may be stationary.
- the angle of the imaging units 44 can be adjustable, which may facilitate capturing images from a target object placed at various angles relative to the vision assistive device 100 .
- the imaging units 44 may be any camera such as, CMOS, CCD, or similar, which is capable of capturing still images or videos.
- the imaging units 44 can be, independently of each other, auto-focused, manually focused, or have a fixed focus with a sufficient focus depth.
- the focusing of the imaging units 44 may be electrically controlled by the control system.
- the imaging units 44 may be supported by a neck 32 .
- the one or more imaging units 44 may be disposed on a telescopic, slidable neck 32 , which can slide in and out of the housing 42 through an opening 28 in the housing 42 .
- the imaging units supported by the neck 32 may be housed within the housing 42 to protect the imaging units 44 from damage when, for example, carrying the visual assistant 100 .
- the neck 32 may be moved to its up-position to bring out the imaging units 44 out of the housing 42 to allow the imaging units 44 to capture images. As shown in FIG. 3A and FIG.
- the neck 32 may be adjusted to position the imaging unit 44 in a desired location relative to the target object from which the images are to be captured.
- the neck 32 may be sufficiently extended to place the imaging unit close to or substantially over a central section of the target object to be read by the vision assistive device 100 to minimize image distortion.
- FIG. 4 , FIG. 5 and FIG. 6 present an exemplary mechanism for moving the neck 32 in and out of the housing 42 .
- a holder 101 a may be disposed on a back wall 102 of the housing 42 to attach one end of a spring (such as a constant force spring, not shown) to the housing 42 , while a holder 101 b may be on the bottom of the neck 32 to tether the other end of the spring 109 to the neck 32 .
- the spring can act to move the neck 32 from the down-position inside the housing 42 to the up-position outside the housing 42 .
- the back wall 102 of the housing 42 may also support a flange 106 with rail cutouts 107 , which are configured to accept pins 104 and 105 protruding from the neck 32 .
- the rail cutouts 107 may guide the movement of the neck 32 back and forth between the down-position and the up-position.
- corresponding gears 108 may be provided on the back wall 102 of the housing 42 and the neck 32 to control the movement of the neck 32 back and forth between the down-position and the up-position, and vice versa.
- a push-push latch 103 may be attached to the bottom of neck 32 .
- the push-push latch 103 may lock the neck 32 in the down-position while the spring is pulling (but not moving) neck 32 up.
- the user can press the neck 32 down, disengaging the push-push latch 32 and allowing the spring to extend the neck tout of the housing 42 .
- the user can push the neck 32 down until the push-push latch 103 locks the neck 32 in the down-position.
- a telescopic neck 32 enables placement of the imaging units 44 away from the housing 42 , in proximity to the center of the target 88 .
- the neck 32 may position the imaging unit 44 a desired distance between the housing 42 and the center of the target 88 .
- the neck 32 may position the imaging unit 44 substantially over the center of the target 88 .
- the vision assistive device 100 may include one or more imaging units 44 integrated with the housing 42 . In this manner, the imaging units 44 may be positioned between a centerline 85 of the target 88 and an edge 87 of the target 88 .
- the imaging units 44 may be positioned substantially over and in alignment with the edge 87 of the target 88 positioned on the surface 86 adjacent the base region 62 of the housing 42 . In some embodiments, the imaging units 44 may be disposed substantially along the wall 41 of the housing 42 . As noted above, the imaging units 44 may be positioned at various angles to the target 88 . In some embodiments, when the imaging unit is positioned over the center of the target 88 , the imaging unit 44 may need to be directed straight down toward the target 88 , that is, with the lens axis substantially perpendicular to the surface 86 and the target 88 .
- the lens axis of the integrated imaging units may be at an acute angle or angle less than 90° relative to a horizontal surface substantially perpendicular to the housing 42 , such, as for example, the surface 86 or the surface of the target 88 from which the image is captured, in instances when such surface of the target is generally planar and substantially parallel to the surface 88 .
- the lens axis of the integrated imaging units may be at an angle between about 5° and about 50° relative to the surface 86 or the target 88 .
- the lens axis of the integrated imaging units may be at an angle between about 10° and about 30° relative to the surface 86 or the target 88 .
- the lens axis of the integrated imaging units may be at an angle between about 15° and about 20° relative to the surface 86 or the target 88 .
- the lens axis may be positioned parallel to the surface 86 .
- the imaging units 44 may be positioned in a fixed location in the top region 64 of the housing 42 .
- the vision assistive device 100 may include a single imaging unit 44 , which may be positioned on the front side of the housing 42 substantially along a centerline of the housing 42 .
- the vision assistive device 100 may include two imaging units 44 a and 44 b .
- the imaging units 44 a and 44 b may be positioned along the top region 64 of the housing 42 and may be spaced away from the side walls of the housing 42 and spaced away from one another.
- each of the imaging units 44 a , 44 b which may comprise of a sensor and a lens, may be down-facing such that their respective viewing area is a respective portion of the target 88 to capture an image of that portion of the target 88 , and the images of the respective portions captured by the imaging units 44 a , 44 b may be subsequently combined, such as by stitching, together by the control system, as will be described below.
- each imaging unit may be responsible for slightly more than half of the target 88 to enable the captured images to overlap for subsequent stitching by the control system.
- the exact position of the imaging units 44 a and 44 b along the top region 64 of the housing 42 may depend on the resolution and field of view of the imaging units 44 a and 44 b .
- each imaging unit 44 a and 44 b may be covering slightly more than one half of the total area of the target 88 .
- each of the imaging units 44 a , 44 b only needs to cover slightly more than one half of the total area of the target 88 , the combined resolution of such system is almost double (with discount for the overlap) the resolution comparing to a similar system with only one imaging unit 44 covering the total area of the target 88 . Accordingly, the higher the number of imaging units, the lower resolution imaging units may be employed.
- the imaging units 44 a and 44 b may be positioned such that the fields of view 111 and 112 of the imaging units 44 a and 44 b , respectively, extend past the outer edges of the target 88 while providing an overlap 113 in the middle of the target 88 .
- the overlap 113 between the fields of view 111 , 112 should be sufficient for the control system to align and stitch together the images captured by the imaging units 44 a , 44 b .
- the overlap area 113 may include sufficient number of features to enable the control system to fully reconstruct the combined image from the imaging units 44 a , 44 b . If the overlap area 113 has sufficient number of features, the process of combining images from the imaging units 44 a and 44 b can be performed.
- the extent of the overlap area 113 may also be adjusted depending on the thickness of the target 88 or range of thickness of target 88 . For example, as the thickness of the target 88 increases, the width of the overlap area 113 may be decreased. It should be noted that the vision assistive device 100 may employ more than two imaging units.
- FIG. 10 illustrates a down-facing imaging unit 44 having a lens 145 and a sensor 144 .
- the lens 145 is generally down-facing to capture an image of the target 88 placed substantially perpendicular to the vision assistive device on the surface, which also supports the vision assistive device 100 .
- the lens 145 has a central axis or a lens axis 147 passing substantially through a center of the lens 145 .
- the central axis or lens axis 147 forms an angle 148 relative to the surface 86 or target 88 .
- the angle 148 is an acute angle.
- the imaging unit 44 has a field of view (FOV) 146 that covers the target 88 , and potentially also covers additional margin to, for example, account for inaccurate positioning of the target or non-standard size of the target.
- FOV field of view
- the distance of the imaging units 44 from the surface 86 and an angle of the axis of the imaging units to the horizontal plane substantially perpendicular to the housing may be a function of the viewing angle (A) of the imaging units 44 , the size of the target 88 , and the number of imaging units used. Both the distance and lens axis angle can be adjusted for maximum performance.
- the lens axis is not at the centerline of the target, but is closer to the edge which is closer to the imaging unit.
- the two imaging units 44 a and 44 b with viewing angle 60° may be positioned approximately 9.3 inches above the surface at an angle of approximately 17.6°, and distanced 5.25 inches from each other to cover a letter sized target with sufficient overlap.
- the vision assistive device 100 may also include a control system 120 for controlling various functionalities of the vision assistive device 100 .
- the control system 120 may receive data from the imaging units corresponding to the captured image, process the data, and output the data to the user.
- the captured image from the target object 88 may be displayed in an enlarged format in video mode (CCTV), showing the captured image magnified. Either multiple imaging units or a single imaging unit of multiple imaging units can be used in this mode. In some embodiments, especially when the image includes a text portion, the captured image may be segmented into text and graphics.
- the text portion may then be further processed, as necessary, such as to reduce blur, skew, and distortion, to prepare the text for optical character recognition (OCR).
- OCR optical character recognition
- the text may be outputted in a visual form on a screen 20 .
- the screen 20 may be integrated into the housing 42 of the vision assistive device 100 .
- an external screen may be employed.
- the control system 120 may include text-to-speech conversion capabilities to enable the vision assistive device 100 to read the OCRed text to the user through a speaker in the housing 42 .
- the vision assistive device 100 may include speakers 26 .
- the control system 120 may also stitch the images captured by multiple imaging units before the image can be analyzed.
- Various softwares for stitching images may be used.
- the image stitching process by the control system 120 can be divided into three main components—1) image registration, 2) calibration, and 3) blending.
- Image registration may involve matching features in a set of images.
- the control system 120 may perform OCR of multiple images captured by multiple imaging units and determine position of lines on each image.
- the processing 120 may determine which lines in the multiple images may be coinciding or overlapping.
- the control system 120 may construct the lines that are intersecting overlap area, thereby reconstruction the original image from multiple captured images.
- the control system 120 may also employ direct alignment methods to search for image alignments that minimize the sum of absolute differences between overlapping pixels. When using direct alignment methods, the control system 120 might first calibrate images to get better results.
- Image calibration aims to minimize differences between an ideal lens models and the imaging unit—lens combination that was used, as well as optical defects and distortions. It also aims at compensating for imperfect lens positioning. Because of the tolerances in lens housing as well as of the top region 64 of the housing 42 , the lens axis angle as well as the same angle in other vertical plane may have tolerances. If feature detection methods were used to register images and absolute positions of the features were recorded and saved, stitching software may use the data for geometric optimization of the images. Image blending may involve executing the adjustments figured out in the calibration stage, combined with remapping of the images to an output projection. Colors may be adjusted between images to compensate for exposure differences.
- the control system 120 may be utilized to calibrate multiple imaging units.
- multiple imaging units may be calibrated using chess board pattern.
- the chess board pattern may be printed on a paper of a size larger then combined field of view (FOV) of the imaging units. The pattern may start on the edge of the target area.
- the chess-board pattern is easily identifiable by the imaging units and may also have an easily identifiable dot that is located in the middle of the target area.
- Performing calibration may enable the control system 120 to determine, among other things, the exact position of the center of the target area on each imaging unit, the exact boundaries of the target area, and the exact overlap of the imaging units. The control system may then use this data to facilitate the stitching process.
- images captured with a single imaging unit may need to be stitched too.
- different parts of the captured image are located at different distances from the imaging unit.
- the distance difference may be sufficiently large to require focusing of the imaging units 44 on these parts of the captured image separately and capturing separate images to get a sufficiently sharp image of the entire target area upon stitching images of object parts located at different distances. Taking such multiple shots can be done without turning the imaging units or the visual assist device 100 .
- the imaging units may capture multiple images of the same FOV to have different parts of the imaging units' FOV in focus separately, with appropriate focus adjustment from one shot to another, which may be controlled by the control system 120 .
- the need for capturing separate images of separate parts of the FOV of an imaging unit and then stitching them together may depend on such FOV geometry and on the shape of the target.
- an imaging unit may be used to capture an image of a flat page such as a flat target object of paper placed on a flat desk surface.
- the imaging unit may be fixed in space, so that the distance from the imaging unit to the page becomes as a function of the pixel location is known. The distance may vary from one part of the FOV to another. For this reason, the focal depth of the imaging unit may be insufficient to image the entire page sharply enough in a single shot. If so, a proximal part of the page may be imaged with the imaging unit focused on it, and, separately, a distant part of the page may be imaged separately according to its own distance from the imaging unit.
- the two images are therefore captured separately by a single imaging unit and then stitched together by the control system. More than two images can be captured and stitched together in a similar manner.
- the imaging unit is integrated with the housing as, for example, shown in FIG. 8 and FIG. 9 , there may be more variation in the distance between the imaging unit and different parts of the target 88 .
- the process for stitching images from a single imaging unit 44 may be utilized to prepare an image of desired quality prior to stitching images from multiple imaging units together.
- the distance between the imaging unit and the center of the target may be smaller than the distance between the imaging unit and the corners.
- control system 120 may be programmed to select multiple focusing.
- the control system 120 can be programmed to analyze the imaging unit-to-object distance distribution within the FOV. Accordingly the control system 120 can give a command to take either a single shot or multiple shots of the FOV, the imaging unit's focus being adjusted to the distance in each direction selected for imaging.
- the control system 120 may also select the number of such shots, their focal distances and the corresponding shapes and sizes of the areas of the FOV to be stitched together upon capturing their images. Such selection by the control system 120 can take into account the inherent known geometry of the visual assist device standing on a flat desk. Focus distance can be measured toward at least one point in the object.
- Control system 120 may be programmed such that the control system 120 can find the optimal focal position to a predefined region of the target, and then assumes that the difference between focal positions between different regions is known.
- the control system 120 may include a processor 121 coupled to various system components by a system bus 110 .
- the system components may include the system memory such as read only memory (ROM) 140 and random access memory (RAM) 150 to the control system 120 .
- Other system memory 130 may be available for use as well.
- the system bus 110 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures.
- a basic input/output (BIOS) stored in ROM 140 or the like, may provide the basic routine that helps to transfer information between elements within the computing device 100 , such as during start-up.
- BIOS basic input/output
- the control system 120 may further include storage devices such as a hard disk drive 160 , a solid state storage, magnetic disk drive, an optical disk drive, tape drive or the like.
- the storage device 160 is connected to the system bus 110 by a drive interface.
- the drives and the associated computer readable media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for the control system 120 .
- Any number of input mechanisms 190 can be connected to the control system 120 .
- the output device 170 can be one or more of a number of output mechanisms known to those of skill in the art, for example, printers, monitors, projectors, speakers, and plotters.
- multimodal systems enable a user to provide multiple types of input to communicate with the control system 120 .
- the communications interface 180 may generally govern and manage the input and system output.
- control system 120 is illustrated as comprising individual functional blocks (including functional blocks labeled as a “processor”).
- the functions these blocks represent may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software.
- the functions of one or more processors may be provided by a single shared processor or multiple processors.
- Illustrative embodiments may comprise microprocessor and/or digital signal processor (DSP) hardware, read-only memory (ROM) for storing software performing the operations discussed below, and random access memory (RAM) for storing results.
- DSP digital signal processor
- ROM read-only memory
- RAM random access memory
- VLSI Very large scale integration
- Embodiments within the scope of the present disclosure may also include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon.
- Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer.
- Such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, solid state or other storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures.
- Computer-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions.
- Computer-executable instructions also include program modules that are executed by computers in stand-alone or network environments.
- program modules include routines, programs, objects, components, and data structures, etc. that perform particular tasks or implement particular abstract data types.
- Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.
- the vision assistive device 100 may include one or more light sources 200 adjacent to the imaging units 44 for illuminating the target 88 .
- the light source may include one or more light emitting diodes (LEDs).
- the light sources 200 may be spaced away from imaging units 44 and may be shared by the imaging units 44 .
- the vision assistive device 100 may also include a user-facing imaging unit 202 , which can be used for imaging the user for such purposes as video telephony.
- the user-facing imaging unit 202 may be configured to be facing the user sitting in front of the vision assistive device 100 . It should of course be understood that the user-facing imaging unit 202 can image any other object, for example, it can serve for surveillance.
- a switch may be provided to activate the user-facing imaging unit 202 , while deactivating other image capturing units 44 . The switch may inform the control system 120 which may then switch activity between the user-facing imaging unit 30 and the other imaging units 44 , and optionally turns an indicator light on to let the user know which imaging unit is active, if any.
- such a switch can alternatively be triggered by an accelerometer sensing the angle between the neck 32 and the horizontal plane.
- a switch does not deactivate any imaging units, but rather selects which of the imaging units, if any, sends its image to be displayed to the user. Multiple switches can be used.
- the vision assistive device 100 may further include one or more analog controls.
- the vision assistive device 100 may include one or more rollers 204 , 206 for controlling, selecting, activating or deactivating, various functionalities of the vision assistive device 100 .
- the control system 120 notes in which direction and by how many degrees the one or more rollers 204 , 206 are rotated, the control system 120 translates the rotation of the one or more rollers 204 , 206 into various functions.
- Examples of such functions include, but not limited to, scrolling the image or text up/down on the screen 20 , in a page-down and page-up or line up and line down manner; moving between menu items, scroll through telephone director or address book to another, scroll through image (for example, roller 204 does vertical scrolling while roller 206 does horizontal scrolling), zooming image in and out, controlling reading speed, hanging font size and similar functions.
- scrolling the image or text up/down on the screen 20 in a page-down and page-up or line up and line down manner
- moving between menu items scroll through telephone director or address book to another, scroll through image (for example, roller 204 does vertical scrolling while roller 206 does horizontal scrolling), zooming image in and out, controlling reading speed, hanging font size and similar functions.
- roller 204 does vertical scrolling while roller 206 does horizontal scrolling
- zooming image in and out controlling reading speed, hanging font size and similar functions.
- 206 icon or clickable images are highlighted on the screen 20 for selection by the user.
- buttons 208 , 210 , 212 , 214 , 216 and 218 for controlling the functionalities of the vision assistive device.
- the control system 120 by its software, assigns (defines the designation of) various functions to the buttons 208 , 210 , 212 , 214 , 216 and 218 .
- buttons 208 and 210 may be assigned a select function for selecting items from a menu.
- Buttons 212 , 214 , 216 and 218 may be located on the top surface of the housing 42 of the vision.
- buttons may be such functions as volume control (some of the buttons can be rotary knobs) of voice/speech output (audio), speed control of voice/speech output (audio), snapshot, mode switching, magnification control, brightness, contrast.
- the vision assistive device 100 may further include a power on/off button 220 .
- one or more rollers 204 , 206 may be used to scroll through the image to select a portion of the text, by the roller or using one of the buttons, that can then be zoomed in or out to a desired size by, for example, using the other roller 204 , 206 .
- buttons may be used to prompt the control system 120 to provide audio output of the selected portion to the user.
- control system 120 of the vision assistive device 100 may enable the vision assistive device 100 to have various functionalities. Following is a description of some examples of the functionalities that may be included in the vision assistive device of the present disclosure.
- the vision assistive device 100 may include one or more of the functionalities discussed below, as well as additional functionalities typically included in similar devices.
- the vision assistive device 100 of the present disclosure can be connectible to the Internet by wire or wireless connection, for browsing websites, emailing, and providing customized services such as customized news, weather forecast, traffic conditions, making police or medical emergency calls with one button push remotely from other rooms using WiFi for example using a bracelet, etc.
- Internet activity can be similar to that by general public, while its control by the user is preferably done via perceived mechanical means, such as large buttons, sliders, roller-wheels and knobs, for example by scrolling up and down through the list of user's favorite websites, TV channels, movies, or email contacts.
- Icons and/or clickable images on screen 20 can be highlighted, one by one in turn, by turning the rollers 204 and/or 206 .
- a highlighted image can then be clicked on by pushing a button.
- a button can be one of buttons 208 and 210 located above in proximity to rollers 204 and/or 206 .
- Emailing can be done by writing by means of speech recognition (speech-to-text conversion by either the vision assistive device 100 or the server) or/and by sending voice-messages.
- the vision assistive device 100 of the present disclosure can be connectible to a dedicated server to receive personalized web services. Services can be personalized, for example, when the vision assistive device 100 connects to a server designated to providing services to this kind of vision assistive device 100 .
- the server may know (from a code provided by the vision assistive device 100 to the server, such as the vision assistive device 100 serial number) which info to provide to the user of the vision assistive device 100 .
- the server may provide weather forecast based on the user residence.
- the server may provide such information based on user's location.
- the server can store images taken by the vision assistive device 100 and make them available to designated persons, such as user's relatives, care-takers or doctors.
- the images can be of the user, captured by user-facing imaging unit 202 , or of a document, captured by the imaging unit 44 .
- Internet navigation or/and navigation within a vision assistive device 100 -dedicated server can be done by the user without either keyboard, mouse, or touch screen. Rather, the user may use perceived mechanical controls such as buttons and roller wheels described above.
- the service types provided by the server can be personalized too, as the designated server identifies the user.
- the user may have a personalized list of movies and/or TV channels to watch on the vision assistive device 100 via the Internet.
- the user may choose to place specific personal medical information on the server for viewing by selected individuals only.
- Email writing can be done by means of speech recognition (speech-to-text conversion by either the vision assistive device 100 or the server) or/and by sending voice-messages.
- the vision assistive device 100 may be configured to output information in a simplified format. For example, long passages of text unrelated to clickable images and phrases can be shrunk or removed. Also or alternatively, small font text can be shrunk or removed.
- the vision assistive device 100 may include a microphone and recording capabilities to allow the user to record reminders to self or others, or other messages. Playback options may be included for the user to retrieve the recorded messages.
- the vision assistive device 100 of the present disclosure can be connectible to a phone line.
- the vision assistive device 100 may be used as a fax and provide customized services such as customized news, weather forecast, traffic conditions, making police or medical emergency calls with one button push remotely from other rooms using Wi-Fi for example using a bracelet, etc.
- Such services can be personalized similarly to Internet services described above.
- a phone line without a broadband Internet access can still provide an old fashioned dial-up Internet access, even if the user does not realize how the vision assistive device 100 is accessing the Internet.
- the vision assistive device 100 can serve for surveillance purposes, either video, still or audio.
- control system 120 can be equipped with software able to distinguish between images (either still or video) of humans from those of animals, such as dog, rabbit, cat, deer, coyote, etc. Switching to either of these modes or to a combination thereof can be done by means of pulling out, pushing in, sliding, or turning a large unit of control as described. Similar mechanical movements can be used for customization of the vision assistive device 100 , in other words, for fixing the modes, applications, or other settings available on this vision assistive device 100 . In this way, perceived analog controls can be used for setting up the vision assistive device 100 , not only for using it.
- the vision assistive device 100 may notify the user, local police department or a security company, phone, email, SMS or any other means.
- the vision assistive device 100 can be programmed to remind the user to take medications or take other actions, possibly on a daily basis.
- the vision assistive device 100 may include a calendar and a clock, as well as a medication administration schedule in control system 120 .
- the schedule program can produce audio or/and visual pill-time signals to let the user know when to take a medication. This function is similar to an alarm clock.
- the pill-time signal can be accompanied by information, such as text or speech.
- the pill-time signal can also go to a caregiver, by phone, email, SMS or any other means.
- the caregiver can then assist the user of the vision assistive device 100 in selecting the proper medication and advise about the proper administration.
- the caregiver can watch the medications and/or the vision assistive device 100 user taking the medication to make sure it is administered properly.
- One way to watch is using a connection via the Internet. Any of the imaging units in the vision assistive device 100 can be used for such watching.
- a dedicated caregiver may be allowed to change the medical schedule in control system 120 over the Internet.
- the vision assistive device 100 itself can assist in choosing the correct medication package, by recognizing the image of the package or on the package, such as a barcode, text, symbol, etc. For example, upon the pill-time signal, the user can bring a pill bottle into the FOV of the imaging unit 44 .
- the control system 120 can use image recognition software to determine whether the bottle is correct for administration of the medication according to the pill-time signal. Then the control system 120 can give positive or negative feedback (audio and/or video) to inform the user whether the package is correct or not.
- the user of the vision assistive device 100 may have a tray of medications, the medications being placed in multiple compartments. Medications in a compartment are intended for administration within a certain time interval, possibly together. Each compartment has a cover. A cover may have a unique identifiable symbol. 1D or 2D barcode is one example of such a symbol.
- the control system 120 may use image recognition software to tell whether the compartment is right judging by whether the correct symbol has disappeared. Alternatively, identification of the compartment is done judging by the location of the compartment in the tray.
- control system 120 can use image recognition software to compare the pills remaining in the compartment, which may be zero pills or more, with the pills that had been there originally. Pill size, color, number, etc. can be used for such comparison. The comparison can tell whether the pill administration has been correct.
- the vision assistive device 100 can further register the compliance of the user with the medication administration schedule.
- the user can be required to press a button to confirm the pill is taken.
- the user may be required to hold an open hand with a pill in it under a imaging unit of the vision assistive device 100 for capturing an image of it before taking the pill.
- the captured image may serve as evidence of the user having taken a pill.
- the captured image may even tell which pill was taken, upon image recognition by either a human or machine.
- the combination of the medical schedule calendar, control system 120 , and the imaging unit in the vision assistive device 100 is essential for follow up to ensure compliance.
- Compliance also adherence or capacitance
- the vision assistive device 100 acts upon user's error.
- the control system 120 can be programmed so that in case the user makes an identifiable error (i.e. takes a wrong pill, has not taken it at all, etc.) the vision assistive device 100 can take appropriate measures, such as inform a caregiver and/or emergency service via email, SMS or automatic phone call, log the occurrence into a trouble list, or repeat the pill-time signal again later.
- the vision assistive device 100 may be programmed to allow the user to play various games using the vision assistive device 100 .
- the imaging unit 44 can be used to capture images of crossword puzzles from paper, with user's hand-written scribbles on them.
- the control system 120 can present the puzzle in a convenient format on the screen 20 .
- Such format can be displaying the question next to the corresponding column/row of the wanted word on the screen 20 .
- the user can point at either the puzzle question or the column/row of interest with a pen or finger.
- the control system 120 can identify such pointing by means of image recognition software, and display the puzzle question next to the corresponding column/row.
- displaying magnified text on the screen 20 can be useful for low vision users, making reading easier.
- the magnified display can be either in video mode (CCTV), showing the captured image as is, or upon OCR (optical character recognition) in an artificial font.
- OCR optical character recognition
- the vision assistive device 100 can read aloud what the printed text says.
- speech output can be useful for blind users, making the vision assistive device 100 a reading machine.
- the vision assistive device 100 of the present disclosure can be equipped with various sensors other than an imaging unit.
- exemplary sensors include, but are not limited to, sensors for monitoring physiological data, such as blood pressure, heart rate, glucose level in blood, body temperature, etc. Additional sensors may include a microphone, smell sensor, fire alarm sensor, smoke detector, ambient temperature sensor, humidity sensor, atmospheric pressure sensor, carbon monoxide sensor, etc.
- the data collected by the sensors can be outputted on the screen 20 .
- audio output can also be used.
- Medication administration schedule stored in the control system 120 can be displayed or read aloud too. Switching to either of these modes of sensing and outputting or to a combination thereof can be done by means of pulling out, pushing in, sliding, or turning a large unit of control as described.
- a carbon monoxide sensor can be built-in within or connected to the vision assistive device 100 (by wire or wirelessly).
- the vision assistive device 100 can alarm the user with audio and/or visual messages.
- Such messages can reach other rooms wirelessly.
- the vision assistive device 100 can send such messages by email or/and phone to designated recipients such as relatives, neighbors, or/and fire department. Turning such a carbon monoxide sensor (or its connection to the vision assistive device 100 ) on and off can be done by means of pulling out, pushing in, sliding, or turning a large unit of control as described.
- a carbon monoxide sensor is just one example.
- Other sensors such as blood pressure, heart rate, glucose level in blood, body temperature, etc., can be used similarly and so can their alarm messages.
- the user can control the vision assistive device 100 by gestures.
- the printed text converted into magnified text on a monitor (for example as a scrolling line), or into speech, is intended for user's consumption.
- the user may wish to have control over the flow of the output text or speech.
- control may involve giving commands similar to what is called in other consumer players “Stop”, “Play”, “Fast-Forward” and “Rewind” commands.
- Commands such as “Zoom In”, “Zoom Out” can also be given by gestures, even though they may not be common in other consumer players.
- the imaging unit is usually in video mode, yet is not monitoring turning pages in book-scanning setting.
- the imaging unit can be used to sense a specific motion or an image that signals to the algorithm that the corresponding command should be executed. For example, moving a hand in a specific direction under the imaging unit can signal one of the above commands. Moving a hand in a different direction under the imaging unit can signal a different command.
- the field of view of the imaging unit can be arranged to have a horizontal arrow that can be rotated by the user around a vertical axis.
- the image-processing algorithm can be pre-programmed to sense the motion and/or direction of the arrow. Such a motion can be detected and a change in the direction of the arrow can be identified as a signal.
- a signal a “gesture”.
- a common software algorithm for the identification of the direction of motion known as “Optical Flow” algorithm, can be utilized for such gesture recognition.
- gesture interpretation can be pre-programmed to depend on the current state of the output flow. For example, gesture interpretation can differ between the states in which 1) the text is being read out (in speech) to the user, 2) the text reading has been stopped, and 3) magnified text is being displayed. For example the gesture of moving a hand from right to left is interpreted as the “Stop” (aka “Pause”) command if the output text or speech is flowing. Yet, the same gesture can be interpreted as “Resume” (aka “Play”) if the flow has already stopped.
- Moving a hand in other manners can signal additional commands. For example, moving a hand back and forth (e.g. right and left), repeatedly, can signify a command, and repeating this movement a preset number of times within a preset time-frame can signify various additional commands.
- Gestures can also be interpreted as commands in modes other than output flow consumption. For example, in Video Mode, a gesture can give a command to change optical zoom or digital magnification. For this purpose, it is desirable to distinguish motion of a hand from other motion, such as motion of printed matter under the imaging unit.
- the software that processes the video stream can recognize shapes of human fingers or the palm of the hand. With this capability, the software can distinguish motion of the user's hands from motion of the printed matter.
- the vision assistive device 100 may include a motion detector to switch on and off various functions of the vision assistive device 100 .
- the vision assistive device may be used for scanning of a book. During scanning of a book, alternating time intervals of motion and no motion can convey the process of turning pages. Such time intervals of motion and no motion can be considered as gestures too, even if the motion direction is irrelevant for the interpretation of the gesture. For example, as a page of a book is being turned, motion is being detected by motion detector software in the control system 120 , while monitoring video stream via an imaging unit that faces printed matter. The detected motion may be either that of a hand or that of printed matter.
- the fact that the page has been turned over and is ready for photographing is detected by the motion detector as the subsequent absence of motion.
- the software interprets the drop as the page having been turned over. This may trigger taking a picture (photographing, capturing a digital image, a shot) and signaling this event to the user. Before the next shot is taken, the detector should see enough motion again and then a drop in motion for a long enough period of time. In this mode (e.g., book scanning), motion in any direction is being monitored, unlike in specific hand gesture recognition during output consumption, where motion in different directions may mean different commands.
- the control system 120 can assign those features to their coordinates (position) in the field of view (FOV) of the imaging unit.
- the images corresponding to a specific coordinate range are not pre-defined a priori but rather assigned to their position after photographing the target.
- One use of these page features is giving commands by covering and uncovering these images, e.g. by hand. This may include hand gestures seen as time sequences of covered and uncovered images (and thus their positions).
- the stored page image can work for this purpose as long as the page remains under the imaging unit with no appreciable shift relative to its location when imaged. The presence of the page with no shift may be monitored.
- the algorithm can search 1) for another page or 2) for the same page with a shift.
- the images seen by the imaging unit can serve as a new predefined recognizable image set as described above.
- moving a hand over such a page can signify various commands depending on the direction of the movement.
- covering a specific portion of such a page with a hand can also signify a command.
- covering a subset of images located around a specific corner of the FO, as viewed from the imaging unit may signal a command that is different from a command signaled by covering a subset of images around a different corner of the FOV.
- Such covering can be achieved by placing printed matter, a hand, or other objects on a subset of images or above it.
- the resulting commands can include the “Stop”, “Play”, “Fast-Forward” and “Rewind” commands, as well as activating the motion-detector mode for scanning a book.
- Time sequences of such covered and uncovered images can be pre-programmed to encode various commands.
- a large number of commands can be encoded by such time sequences.
- Moving a hand above the surface of images in a specific manner can signal commands by way of covering and uncovering the images in various order (sequences). For example, moving a hand back and forth (e.g. right and left) can signify a command. Repeating this movement a preset number of times within a preset time-frame can signify various additional commands.
- the shape of a hand can be used to differentiate such hand gestures from movement of printed matter over the surface. Such shape can be indicated by the silhouette of the set of images covered at any single time.
- image recognition algorithms can be used for the purpose of recognizing hands, fingers, etc.
- a vision assistive device includes a housing having a base for positioning the housing on a surface; one or more imaging units disposed along a top portion of the housing, each imaging unit being angled downward so a central axis of the imaging unit forms an acute angle with the surface so the imaging unit captures a target image from a target object; and a control system accommodated within the housing, the control system being connected to the one or more imaging units to process the target image captured by each imaging unit and to output the target image to a user.
- a vision assistive device includes a housing having a base for positioning the housing on a surface; one or more imaging units configured to capture a target image from a target object positioned substantially perpendicular to the housing; a screen integrated with the housing; a control system accommodated within the housing, the control system being connected to the imaging units to process data corresponding to the image captured by the imaging units and to output the target image to a user on the screen.
- a method for providing vision assistance includes disposing a housing substantially perpendicular to a surface, the housing including one or more imaging units and a control system; capturing with the one or more imaging units a target image from a target object placed on the surface adjacent to the housing; and processing the captured target image and outputting the target image on a screen integrated with the housing.
Abstract
Vision assistive device and user interface are disclosed. In some embodiments, a vision assistive device includes a housing having a base for positioning the housing on a surface; one or more imaging units disposed along a top portion of the housing, each imaging unit being angled downward so a central axis of the imaging unit forms an acute angle with the surface so the imaging unit captures a target image from a target object; and a control system accommodated within the housing, the control system being connected to the one or more imaging units to process the target image captured by each imaging unit and to output the target image to a user.
Description
- This application is a continuation application of U.S. application Ser. No. 14/169,736, filed on Jan. 31, 2014, entitled “Vision Assistive Devices and User Interfaces,” now U.S. Pat. No. 9,449,531, issued Sep. 20, 2016, which is a continuation application of U.S. application Ser. No. 13/829,727 filed on Mar. 14, 2013, entitled “Vision Assistive Devices and User Interfaces,” now U.S. Pat. No. 8,681,268, issued Mar. 25, 2014, which claims the benefit of and priority to U.S. Provisional Application No. 61/688,909, filed on May 24, 2012. All of these applications are incorporated herein by reference in their entireties for all purposes.
- Field of the Invention
- The present disclosure relates to vision assistive devices, and in particular, to free-standing vision assistive devices with one or more imaging units.
- Description of the Background Art
- According to some estimates, nearly 2.9 million Americans are visually impaired. Many more have trouble reading due to other conditions, such as stroke or traumatic brain injury. The prevalence of vision impairment increases rapidly in the later years, particularly after age 75. Even people with the least degree of vision impairment may still face challenges in everyday life as typical everyday activities such as reading a newspaper become difficult if not impossible.
- There are a number of vision assistance devices on the market to assist those whose ability to read is impaired by visual difficulties. However, one problem with the current devices is that the current devices require complex set up before they can be used. Moreover, operating current vision assistance devices can be problematic for a visually impaired person due to the heavy reliance of the current vision assistance devices on digital controls, such as a keypad, keyboard, a mouse, or a touch screen. Accordingly, there is still a need for a vision assistive device that is easy to set up and operate.
- Vision assistive devices and user interfaces are disclosed. In some aspects, there is provided a vision assistive device that includes a housing having a base for positioning the housing on a surface; one or more imaging units disposed along a top portion of the housing, each imaging unit being angled downward so a central axis of the imaging unit forms an acute angle with the surface so the imaging unit captures a target image from a target object; and a control system accommodated within the housing, the control system being connected to the one or more imaging units to process the target image captured by each imaging unit and to output the target image to a user.
- In some aspects, there is provided a vision assistive device that includes a housing having a base for positioning the housing on a surface; one or more imaging units configured to capture a target image from a target object positioned substantially perpendicular to the housing; a screen integrated with the housing; a control system accommodated within the housing, the control system being connected to the imaging units to process data corresponding to the image captured by the imaging units and to output the target image to a user on the screen.
- In some aspects, there is provided a method for providing vision assistance that includes disposing a housing substantially perpendicular to a surface, the housing including one or more imaging units and a control system; capturing with the one or more imaging units a target image from a target object placed on the surface adjacent to the housing; and processing the captured target image and outputting the target image on a screen integrated with the housing.
- The presently disclosed embodiments will be further explained with reference to the attached drawings. The drawings shown are not necessarily to scale, with emphasis instead generally being placed upon illustrating the principles of the presently disclosed embodiments.
-
FIG. 1 is a prospective view on an embodiment of a vision assistive device of the present disclosure. -
FIG. 2 illustrates an embodiment of a vision assistive device of the present disclosure placed on a surface for capturing an image from a target. -
FIG. 3 is a perspective view of the front of the magnifying apparatus.FIG. 3A ,FIG. 3B , andFIG. 3C illustrate an embodiment of a vision assistive device of the present disclosure having a slidable neck. -
FIG. 4 ,FIG. 5 , andFIG. 6 illustrate an embodiment mechanism for operating a slidable neck. -
FIG. 7 illustrates various positions of imaging units over a target by a vision assistive device of the present disclosure. -
FIG. 8 illustrates an embodiment of a vision assistive device of the present disclosure with integrated imaging units. -
FIG. 9 andFIG. 10 illustrate various locations of imaging units relative to a target. -
FIG. 11 illustrates an embodiment of a control system for controlling functionalities of a vision assistive device of the present disclosure. -
FIG. 12 is a front view on an embodiment of a vision assistive device of the present disclosure showing various control features of the vision assistive device; - While the above-identified drawings set forth presently disclosed embodiments, other embodiments are also contemplated, as noted in the discussion. This disclosure presents illustrative embodiments by way of representation and not limitation. Numerous other modifications and embodiments can be devised by those skilled in the art which fall within the scope and spirit of the principles of the presently disclosed embodiments.
- The present disclosure provides vision assistive devices, systems and methods of use of vision assistive devices and systems.
- In reference to
FIG. 1 , a visionassistive device 100 of the present disclosure includes ahousing 42. Thehousing 42 may be made of separate parts integrated into a single unit. Thehousing 42 may include abase region 62 and atop region 64 connected by afront wall 66,back wall 68, and twoside walls housing 42 may be continuous and, together with thebase region 62 and thetop region 64, define an inner volume for accommodating therein electrical components (not shown) for carrying out functionalities of the visionassistive device 100, as is described below. - In reference to
FIG. 2 , thehousing 42 can be free standing. In some embodiments, thebase region 62 of thehousing 42 may be designed to enable the visionassistive device 100 to stand unsupported or unassisted on asurface 86 in operation. In this manner, the visionassistive device 100 can simply be placed on asurface 86, without any further manipulations. In some embodiments, thebase region 62 of thehousing 42 may be substantially planar to enable the visionassistive device 100 to stand unassisted on thesurface 86, substantially perpendicular to the surface. In this manner, atarget 88 with a target image (text, indicia, graphics or other images) that the user needs thevision assistance device 100 to capture from thetarget 88 can be placed on thesurface 86 adjacent to the visionassistive device 100 for capturing the images on the target object of paper by the visionassistive device 100, and subsequently communicating the images to the user. In some embodiments, thetarget 88 may be placed such that one edge of the target is along thehousing 42. The surface of thetarget 88 on which the image is located is preferably substantially planar, however, other shapes may also be accommodated. In some embodiments, thehousing 42 may include a foldable ortelescopic handle 43 to facilitate carrying the visionassistive device 100. - In reference to
FIG. 2 , thehousing 42 may include a plurality ofports 52 for connecting the electrical components housed inside thehousing 42 to external devices and power sources. In some embodiments, thehousing 42 may include one or more USB, VGA, HDMI, Ethernet, and other ports for connecting the visionassistive device 100 to external device, including, but not limited to, external data storage, phones, computers, speakers, televisions, monitors, printers and similar devices. Thehousing 42 may further include one or more slots for insertion of a smart card or memory card. Furthermore, thehousing 42 can include wireless interfaces, such as WiFi or Bluetooth. Thehousing 42 may also support a plurality of control buttons to allow the user to control one or more functions of the visionassistive device 100, which control buttons and functions are described in details below. - In reference to
FIG. 1 andFIG. 2 , the visionassistive device 100 may include one ormore imaging units 44 disposed in thetop region 64 of thehousing 42. The number of imaging units may vary depending on, for example, the type of the imaging units employed or the size or shape of thetarget 88. In some embodiments, the visionassistive device 100 may include a single imaging unit. In other embodiments, the visionassistive device 100 may include two imaging units. In yet other embodiments, the visionassistive device 100 may include more than two imaging units. - In some embodiments, the
imaging units 44 may be generally down-facing (but not necessarily pointed straight down) to be able to capture the images from thetarget 88 placed substantially perpendicular to the visionassistive device 100, which is placed, horizontally on thesurface 86 on which the visionassistive device 100 is placed. In some embodiments, theimaging units 44 may be oriented relative to thetarget 88 so the axis of the viewing lens of theimaging units 44 is substantially perpendicular to the surface of thetarget 88. In other embodiments, theimaging units 44 may be oriented relative to thetarget 88 so the axis of the viewing lens of theimaging units 44 is at an acute angle to thesurface 86. In some embodiments, theimaging unit 44 is oriented with the lens axis at an acute angle to thesurface 86 and the lens directing on the center or near the center of thetarget 88 placed on thesurface 86. In some embodiments, theimaging units 44 may be stationary. In other embodiments, the angle of theimaging units 44 can be adjustable, which may facilitate capturing images from a target object placed at various angles relative to the visionassistive device 100. Theimaging units 44 may be any camera such as, CMOS, CCD, or similar, which is capable of capturing still images or videos. Theimaging units 44 can be, independently of each other, auto-focused, manually focused, or have a fixed focus with a sufficient focus depth. In some embodiments, the focusing of theimaging units 44 may be electrically controlled by the control system. In some embodiments, theimaging units 44 may be supported by aneck 32. - In reference to
FIG. 3A ,FIG. 3B andFIG. 3C , in some embodiments, the one ormore imaging units 44 may be disposed on a telescopic,slidable neck 32, which can slide in and out of thehousing 42 through anopening 28 in thehousing 42. When theneck 32 is its down-position, the imaging units supported by theneck 32 may be housed within thehousing 42 to protect theimaging units 44 from damage when, for example, carrying thevisual assistant 100. In operation, theneck 32 may be moved to its up-position to bring out theimaging units 44 out of thehousing 42 to allow theimaging units 44 to capture images. As shown inFIG. 3A andFIG. 3B , theneck 32 may be adjusted to position theimaging unit 44 in a desired location relative to the target object from which the images are to be captured. In some embodiments, as shown inFIG. 3C , theneck 32 may be sufficiently extended to place the imaging unit close to or substantially over a central section of the target object to be read by the visionassistive device 100 to minimize image distortion. -
FIG. 4 ,FIG. 5 andFIG. 6 present an exemplary mechanism for moving theneck 32 in and out of thehousing 42. Aholder 101 a may be disposed on aback wall 102 of thehousing 42 to attach one end of a spring (such as a constant force spring, not shown) to thehousing 42, while a holder 101 b may be on the bottom of theneck 32 to tether the other end of the spring 109 to theneck 32. When activated, the spring can act to move theneck 32 from the down-position inside thehousing 42 to the up-position outside thehousing 42. Theback wall 102 of thehousing 42 may also support aflange 106 withrail cutouts 107, which are configured to acceptpins neck 32. In this manner, therail cutouts 107 may guide the movement of theneck 32 back and forth between the down-position and the up-position. Additionally or alternatively, correspondinggears 108 may be provided on theback wall 102 of thehousing 42 and theneck 32 to control the movement of theneck 32 back and forth between the down-position and the up-position, and vice versa. As shown inFIG. 5 , in some embodiments, a push-push latch 103 may be attached to the bottom ofneck 32. When locked (engaged), the push-push latch 103 may lock theneck 32 in the down-position while the spring is pulling (but not moving)neck 32 up. To extend theneck 32 out of thehousing 42, the user can press theneck 32 down, disengaging the push-push latch 32 and allowing the spring to extend the neck tout of thehousing 42. To collapse the neck from the up-position, the user can push theneck 32 down until the push-push latch 103 locks theneck 32 in the down-position. - In reference to
FIG. 7 , as described above, atelescopic neck 32 enables placement of theimaging units 44 away from thehousing 42, in proximity to the center of thetarget 88. In some embodiments, theneck 32 may position theimaging unit 44 a desired distance between thehousing 42 and the center of thetarget 88. In some embodiments, theneck 32 may position theimaging unit 44 substantially over the center of thetarget 88. Alternatively or additionally, the visionassistive device 100 may include one ormore imaging units 44 integrated with thehousing 42. In this manner, theimaging units 44 may be positioned between acenterline 85 of thetarget 88 and anedge 87 of thetarget 88. In some embodiments, theimaging units 44 may be positioned substantially over and in alignment with theedge 87 of thetarget 88 positioned on thesurface 86 adjacent thebase region 62 of thehousing 42. In some embodiments, theimaging units 44 may be disposed substantially along the wall 41 of thehousing 42. As noted above, theimaging units 44 may be positioned at various angles to thetarget 88. In some embodiments, when the imaging unit is positioned over the center of thetarget 88, theimaging unit 44 may need to be directed straight down toward thetarget 88, that is, with the lens axis substantially perpendicular to thesurface 86 and thetarget 88. In some embodiments, however, when theimaging units 44 are integrated with the housing, the lens axis of the integrated imaging units may be at an acute angle or angle less than 90° relative to a horizontal surface substantially perpendicular to thehousing 42, such, as for example, thesurface 86 or the surface of thetarget 88 from which the image is captured, in instances when such surface of the target is generally planar and substantially parallel to thesurface 88. In some embodiments, the lens axis of the integrated imaging units may be at an angle between about 5° and about 50° relative to thesurface 86 or thetarget 88. In some embodiments, the lens axis of the integrated imaging units may be at an angle between about 10° and about 30° relative to thesurface 86 or thetarget 88. In some embodiments, the lens axis of the integrated imaging units may be at an angle between about 15° and about 20° relative to thesurface 86 or thetarget 88. In some embodiments, the lens axis may be positioned parallel to thesurface 86. - In some embodiments, the
imaging units 44 may be positioned in a fixed location in thetop region 64 of thehousing 42. For example, the visionassistive device 100 may include asingle imaging unit 44, which may be positioned on the front side of thehousing 42 substantially along a centerline of thehousing 42. - In reference to
FIG. 8 , in some embodiments, the visionassistive device 100 may include twoimaging units imaging units top region 64 of thehousing 42 and may be spaced away from the side walls of thehousing 42 and spaced away from one another. As shown inFIG. 10 , each of theimaging units target 88 to capture an image of that portion of thetarget 88, and the images of the respective portions captured by theimaging units target 88 to enable the captured images to overlap for subsequent stitching by the control system. The exact position of theimaging units top region 64 of thehousing 42 may depend on the resolution and field of view of theimaging units imaging unit target 88. Because in the embodiments with multiple imaging units, each of theimaging units target 88, the combined resolution of such system is almost double (with discount for the overlap) the resolution comparing to a similar system with only oneimaging unit 44 covering the total area of thetarget 88. Accordingly, the higher the number of imaging units, the lower resolution imaging units may be employed. - In reference to
FIG. 9 , generally, theimaging units view imaging units target 88 while providing anoverlap 113 in the middle of thetarget 88. Theoverlap 113 between the fields ofview imaging units overlap area 113 may include sufficient number of features to enable the control system to fully reconstruct the combined image from theimaging units overlap area 113 has sufficient number of features, the process of combining images from theimaging units overlap area 113 may also be adjusted depending on the thickness of thetarget 88 or range of thickness oftarget 88. For example, as the thickness of thetarget 88 increases, the width of theoverlap area 113 may be decreased. It should be noted that the visionassistive device 100 may employ more than two imaging units. -
FIG. 10 illustrates a down-facingimaging unit 44 having alens 145 and asensor 144. Thelens 145 is generally down-facing to capture an image of thetarget 88 placed substantially perpendicular to the vision assistive device on the surface, which also supports the visionassistive device 100. Thelens 145 has a central axis or alens axis 147 passing substantially through a center of thelens 145. The central axis orlens axis 147 forms anangle 148 relative to thesurface 86 ortarget 88. As noted above, in some embodiments, theangle 148 is an acute angle. Theimaging unit 44 has a field of view (FOV) 146 that covers thetarget 88, and potentially also covers additional margin to, for example, account for inaccurate positioning of the target or non-standard size of the target. To obtain a desirable FOV, the viewing angle of the imaging unit, its distance away from the target, and its angle are taken into account. The distance of theimaging units 44 from thesurface 86 and an angle of the axis of the imaging units to the horizontal plane substantially perpendicular to the housing (for example, surface 86) may be a function of the viewing angle (A) of theimaging units 44, the size of thetarget 88, and the number of imaging units used. Both the distance and lens axis angle can be adjusted for maximum performance. For example, decreasing the axis angle may improve effective resolution. As a result, the lens axis is not at the centerline of the target, but is closer to the edge which is closer to the imaging unit. By way of a non-limiting example, when employing twoimaging units imaging units - In reference to
FIG. 11 , the visionassistive device 100 may also include acontrol system 120 for controlling various functionalities of the visionassistive device 100. In some embodiments, thecontrol system 120 may receive data from the imaging units corresponding to the captured image, process the data, and output the data to the user. In some embodiments, the captured image from thetarget object 88 may be displayed in an enlarged format in video mode (CCTV), showing the captured image magnified. Either multiple imaging units or a single imaging unit of multiple imaging units can be used in this mode. In some embodiments, especially when the image includes a text portion, the captured image may be segmented into text and graphics. The text portion may then be further processed, as necessary, such as to reduce blur, skew, and distortion, to prepare the text for optical character recognition (OCR). Upon applying OCR, the text may be outputted in a visual form on ascreen 20. As shown in, for example,FIG. 9 , in some embodiments, thescreen 20 may be integrated into thehousing 42 of the visionassistive device 100. Additionally or alternatively, an external screen may be employed. In some embodiments, thecontrol system 120 may include text-to-speech conversion capabilities to enable the visionassistive device 100 to read the OCRed text to the user through a speaker in thehousing 42. To deliver audio output, the visionassistive device 100 may includespeakers 26. - In the embodiments with multiple imaging units, the
control system 120 may also stitch the images captured by multiple imaging units before the image can be analyzed. Various softwares for stitching images may be used. In general, in some embodiments, the image stitching process by thecontrol system 120 can be divided into three main components—1) image registration, 2) calibration, and 3) blending. Image registration may involve matching features in a set of images. Thecontrol system 120 may perform OCR of multiple images captured by multiple imaging units and determine position of lines on each image. Next, based on calibration, theprocessing 120 may determine which lines in the multiple images may be coinciding or overlapping. Based on overlap data, thecontrol system 120 may construct the lines that are intersecting overlap area, thereby reconstruction the original image from multiple captured images. Thecontrol system 120 may also employ direct alignment methods to search for image alignments that minimize the sum of absolute differences between overlapping pixels. When using direct alignment methods, thecontrol system 120 might first calibrate images to get better results. - Image calibration aims to minimize differences between an ideal lens models and the imaging unit—lens combination that was used, as well as optical defects and distortions. It also aims at compensating for imperfect lens positioning. Because of the tolerances in lens housing as well as of the
top region 64 of thehousing 42, the lens axis angle as well as the same angle in other vertical plane may have tolerances. If feature detection methods were used to register images and absolute positions of the features were recorded and saved, stitching software may use the data for geometric optimization of the images. Image blending may involve executing the adjustments figured out in the calibration stage, combined with remapping of the images to an output projection. Colors may be adjusted between images to compensate for exposure differences. - In some embodiments, the
control system 120 may be utilized to calibrate multiple imaging units. By way of a non-limiting example, multiple imaging units may be calibrated using chess board pattern. The chess board pattern may be printed on a paper of a size larger then combined field of view (FOV) of the imaging units. The pattern may start on the edge of the target area. The chess-board pattern is easily identifiable by the imaging units and may also have an easily identifiable dot that is located in the middle of the target area. Performing calibration may enable thecontrol system 120 to determine, among other things, the exact position of the center of the target area on each imaging unit, the exact boundaries of the target area, and the exact overlap of the imaging units. The control system may then use this data to facilitate the stitching process. - In some embodiments, images captured with a single imaging unit may need to be stitched too. For example, different parts of the captured image are located at different distances from the imaging unit. The distance difference may be sufficiently large to require focusing of the
imaging units 44 on these parts of the captured image separately and capturing separate images to get a sufficiently sharp image of the entire target area upon stitching images of object parts located at different distances. Taking such multiple shots can be done without turning the imaging units or thevisual assist device 100. Rather, the imaging units may capture multiple images of the same FOV to have different parts of the imaging units' FOV in focus separately, with appropriate focus adjustment from one shot to another, which may be controlled by thecontrol system 120. The need for capturing separate images of separate parts of the FOV of an imaging unit and then stitching them together may depend on such FOV geometry and on the shape of the target. - By way of a non-limiting example, an imaging unit may be used to capture an image of a flat page such as a flat target object of paper placed on a flat desk surface. The imaging unit may be fixed in space, so that the distance from the imaging unit to the page becomes as a function of the pixel location is known. The distance may vary from one part of the FOV to another. For this reason, the focal depth of the imaging unit may be insufficient to image the entire page sharply enough in a single shot. If so, a proximal part of the page may be imaged with the imaging unit focused on it, and, separately, a distant part of the page may be imaged separately according to its own distance from the imaging unit. The two images are therefore captured separately by a single imaging unit and then stitched together by the control system. More than two images can be captured and stitched together in a similar manner. When the imaging unit is integrated with the housing as, for example, shown in
FIG. 8 andFIG. 9 , there may be more variation in the distance between the imaging unit and different parts of thetarget 88. Accordingly, in some embodiments, the process for stitching images from asingle imaging unit 44 may be utilized to prepare an image of desired quality prior to stitching images from multiple imaging units together. Moreover, even in case of the device with asingle imaging unit 44 adjacent to or over thecenterline 85, the distance between the imaging unit and the center of the target may be smaller than the distance between the imaging unit and the corners. - In some embodiments, the
control system 120 may be programmed to select multiple focusing. Thecontrol system 120 can be programmed to analyze the imaging unit-to-object distance distribution within the FOV. Accordingly thecontrol system 120 can give a command to take either a single shot or multiple shots of the FOV, the imaging unit's focus being adjusted to the distance in each direction selected for imaging. Thecontrol system 120 may also select the number of such shots, their focal distances and the corresponding shapes and sizes of the areas of the FOV to be stitched together upon capturing their images. Such selection by thecontrol system 120 can take into account the inherent known geometry of the visual assist device standing on a flat desk. Focus distance can be measured toward at least one point in the object. For an object such as a stack of pages, both the inherent known geometry and possibly at least one focus distance measurement can be taken into account.Control system 120 may be programmed such that thecontrol system 120 can find the optimal focal position to a predefined region of the target, and then assumes that the difference between focal positions between different regions is known. - Referring back to
FIG. 11 , thecontrol system 120 may include aprocessor 121 coupled to various system components by asystem bus 110. The system components may include the system memory such as read only memory (ROM) 140 and random access memory (RAM) 150 to thecontrol system 120.Other system memory 130 may be available for use as well. Thesystem bus 110 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. A basic input/output (BIOS) stored inROM 140 or the like, may provide the basic routine that helps to transfer information between elements within thecomputing device 100, such as during start-up. Thecontrol system 120 may further include storage devices such as ahard disk drive 160, a solid state storage, magnetic disk drive, an optical disk drive, tape drive or the like. Thestorage device 160 is connected to thesystem bus 110 by a drive interface. The drives and the associated computer readable media provide nonvolatile storage of computer readable instructions, data structures, program modules and other data for thecontrol system 120. - Any number of
input mechanisms 190, external drives, devices connected to ports, USB devices, such as a microphone for speech, touch-sensitive screen for gesture or graphical input, keyboard, buttons, imaging unit, mouse, motion input, speech and so forth can be connected to thecontrol system 120. Theoutput device 170 can be one or more of a number of output mechanisms known to those of skill in the art, for example, printers, monitors, projectors, speakers, and plotters. - In various embodiments, multimodal systems enable a user to provide multiple types of input to communicate with the
control system 120. Thecommunications interface 180 may generally govern and manage the input and system output. - It should be noted that the
control system 120 is illustrated as comprising individual functional blocks (including functional blocks labeled as a “processor”). The functions these blocks represent may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software. For example the functions of one or more processors may be provided by a single shared processor or multiple processors. (Use of the term “processor” should not be construed to refer exclusively to hardware capable of executing software.) Illustrative embodiments may comprise microprocessor and/or digital signal processor (DSP) hardware, read-only memory (ROM) for storing software performing the operations discussed below, and random access memory (RAM) for storing results. Very large scale integration (VLSI) hardware embodiments, as well as custom VLSI or FPGA circuitry in combination with a general purpose DSP circuit, may also be provided. - Embodiments within the scope of the present disclosure may also include computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, solid state or other storage devices, or any other medium which can be used to carry or store desired program code means in the form of computer-executable instructions or data structures. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or combination thereof) to a computer, the computer properly views the connection as a computer-readable medium. Thus, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of the computer-readable media.
- Computer-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions. Computer-executable instructions also include program modules that are executed by computers in stand-alone or network environments. Generally, program modules include routines, programs, objects, components, and data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of the program code means for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps.
- In reference to
FIG. 12 , in some embodiments, the visionassistive device 100 may include one or morelight sources 200 adjacent to theimaging units 44 for illuminating thetarget 88. In some embodiments, the light source may include one or more light emitting diodes (LEDs). In some embodiments, thelight sources 200 may be spaced away from imagingunits 44 and may be shared by theimaging units 44. - The vision
assistive device 100 may also include a user-facingimaging unit 202, which can be used for imaging the user for such purposes as video telephony. The user-facingimaging unit 202 may be configured to be facing the user sitting in front of the visionassistive device 100. It should of course be understood that the user-facingimaging unit 202 can image any other object, for example, it can serve for surveillance. A switch may be provided to activate the user-facingimaging unit 202, while deactivating otherimage capturing units 44. The switch may inform thecontrol system 120 which may then switch activity between the user-facing imaging unit 30 and theother imaging units 44, and optionally turns an indicator light on to let the user know which imaging unit is active, if any. In some embodiments, such a switch can alternatively be triggered by an accelerometer sensing the angle between theneck 32 and the horizontal plane. Optionally, such a switch does not deactivate any imaging units, but rather selects which of the imaging units, if any, sends its image to be displayed to the user. Multiple switches can be used. - The vision
assistive device 100 may further include one or more analog controls. For example, the visionassistive device 100 may include one ormore rollers assistive device 100. When the user rotates the one ormore rollers control system 120 notes in which direction and by how many degrees the one ormore rollers control system 120 translates the rotation of the one ormore rollers screen 20, in a page-down and page-up or line up and line down manner; moving between menu items, scroll through telephone director or address book to another, scroll through image (for example,roller 204 does vertical scrolling whileroller 206 does horizontal scrolling), zooming image in and out, controlling reading speed, hanging font size and similar functions. In some embodiments, as the user rotates the one ormore rollers screen 20 for selection by the user. - Other analog controls of the vision
assistive device 100 may include a plurality ofbuttons control system 120 by its software, assigns (defines the designation of) various functions to thebuttons buttons Buttons housing 42 of the vision. These buttons may be such functions as volume control (some of the buttons can be rotary knobs) of voice/speech output (audio), speed control of voice/speech output (audio), snapshot, mode switching, magnification control, brightness, contrast. The visionassistive device 100 may further include a power on/offbutton 220. - In some embodiments, as noted above, one or
more rollers other roller control system 120 to provide audio output of the selected portion to the user. - As noted above, the
control system 120 of the visionassistive device 100 may enable the visionassistive device 100 to have various functionalities. Following is a description of some examples of the functionalities that may be included in the vision assistive device of the present disclosure. The visionassistive device 100 may include one or more of the functionalities discussed below, as well as additional functionalities typically included in similar devices. - In some embodiments, the vision
assistive device 100 of the present disclosure can be connectible to the Internet by wire or wireless connection, for browsing websites, emailing, and providing customized services such as customized news, weather forecast, traffic conditions, making police or medical emergency calls with one button push remotely from other rooms using WiFi for example using a bracelet, etc. Internet activity can be similar to that by general public, while its control by the user is preferably done via perceived mechanical means, such as large buttons, sliders, roller-wheels and knobs, for example by scrolling up and down through the list of user's favorite websites, TV channels, movies, or email contacts. Icons and/or clickable images onscreen 20 can be highlighted, one by one in turn, by turning therollers 204 and/or 206. A highlighted image can then be clicked on by pushing a button. Such a button can be one ofbuttons rollers 204 and/or 206. Emailing can be done by writing by means of speech recognition (speech-to-text conversion by either the visionassistive device 100 or the server) or/and by sending voice-messages. - In some embodiments, the vision
assistive device 100 of the present disclosure can be connectible to a dedicated server to receive personalized web services. Services can be personalized, for example, when the visionassistive device 100 connects to a server designated to providing services to this kind of visionassistive device 100. The server may know (from a code provided by the visionassistive device 100 to the server, such as the visionassistive device 100 serial number) which info to provide to the user of the visionassistive device 100. For example, the server may provide weather forecast based on the user residence. Similarly, if the user is interested in train or bus schedule, the server may provide such information based on user's location. In some embodiments, the server can store images taken by the visionassistive device 100 and make them available to designated persons, such as user's relatives, care-takers or doctors. For example, the images can be of the user, captured by user-facingimaging unit 202, or of a document, captured by theimaging unit 44. In some embodiments, Internet navigation or/and navigation within a vision assistive device 100-dedicated server can be done by the user without either keyboard, mouse, or touch screen. Rather, the user may use perceived mechanical controls such as buttons and roller wheels described above. - In some embodiments, the service types provided by the server can be personalized too, as the designated server identifies the user. For example, the user may have a personalized list of movies and/or TV channels to watch on the vision
assistive device 100 via the Internet. The user may choose to place specific personal medical information on the server for viewing by selected individuals only. Email writing can be done by means of speech recognition (speech-to-text conversion by either the visionassistive device 100 or the server) or/and by sending voice-messages. - In some embodiments, the vision
assistive device 100 may be configured to output information in a simplified format. For example, long passages of text unrelated to clickable images and phrases can be shrunk or removed. Also or alternatively, small font text can be shrunk or removed. - In some embodiments, the vision
assistive device 100 may include a microphone and recording capabilities to allow the user to record reminders to self or others, or other messages. Playback options may be included for the user to retrieve the recorded messages. - In some embodiments, the vision
assistive device 100 of the present disclosure can be connectible to a phone line. In this manner, the visionassistive device 100 may be used as a fax and provide customized services such as customized news, weather forecast, traffic conditions, making police or medical emergency calls with one button push remotely from other rooms using Wi-Fi for example using a bracelet, etc. Such services can be personalized similarly to Internet services described above. A phone line without a broadband Internet access can still provide an old fashioned dial-up Internet access, even if the user does not realize how the visionassistive device 100 is accessing the Internet. - In some embodiments, the vision
assistive device 100 can serve for surveillance purposes, either video, still or audio. For surveillance, such as for burglary or break-in alarm purposes,control system 120 can be equipped with software able to distinguish between images (either still or video) of humans from those of animals, such as dog, rabbit, cat, deer, coyote, etc. Switching to either of these modes or to a combination thereof can be done by means of pulling out, pushing in, sliding, or turning a large unit of control as described. Similar mechanical movements can be used for customization of the visionassistive device 100, in other words, for fixing the modes, applications, or other settings available on this visionassistive device 100. In this way, perceived analog controls can be used for setting up the visionassistive device 100, not only for using it. In some embodiments, upon detection of an alarming event, the visionassistive device 100 may notify the user, local police department or a security company, phone, email, SMS or any other means. - In some embodiments, the vision
assistive device 100 can be programmed to remind the user to take medications or take other actions, possibly on a daily basis. For this purpose, the visionassistive device 100 may include a calendar and a clock, as well as a medication administration schedule incontrol system 120. The schedule program can produce audio or/and visual pill-time signals to let the user know when to take a medication. This function is similar to an alarm clock. The pill-time signal can be accompanied by information, such as text or speech. - In some embodiments, the pill-time signal can also go to a caregiver, by phone, email, SMS or any other means. The caregiver can then assist the user of the vision
assistive device 100 in selecting the proper medication and advise about the proper administration. The caregiver can watch the medications and/or the visionassistive device 100 user taking the medication to make sure it is administered properly. One way to watch is using a connection via the Internet. Any of the imaging units in the visionassistive device 100 can be used for such watching. A dedicated caregiver may be allowed to change the medical schedule incontrol system 120 over the Internet. - In some embodiments, the vision
assistive device 100 itself can assist in choosing the correct medication package, by recognizing the image of the package or on the package, such as a barcode, text, symbol, etc. For example, upon the pill-time signal, the user can bring a pill bottle into the FOV of theimaging unit 44. Thecontrol system 120 can use image recognition software to determine whether the bottle is correct for administration of the medication according to the pill-time signal. Then thecontrol system 120 can give positive or negative feedback (audio and/or video) to inform the user whether the package is correct or not. - In some embodiments, the user of the vision
assistive device 100 may have a tray of medications, the medications being placed in multiple compartments. Medications in a compartment are intended for administration within a certain time interval, possibly together. Each compartment has a cover. A cover may have a unique identifiable symbol. 1D or 2D barcode is one example of such a symbol. When a pill-time signal goes off, the user should put the tray into the FOV ofimaging unit 44 and then open the compartment believed to contain the medication according to the pill-time signal. Thecontrol system 120 may use image recognition software to tell whether the compartment is right judging by whether the correct symbol has disappeared. Alternatively, identification of the compartment is done judging by the location of the compartment in the tray. Then the visionassistive device 100 informs the user whether the open compartment is the correct one. After the user takes pills from the compartment,control system 120 can use image recognition software to compare the pills remaining in the compartment, which may be zero pills or more, with the pills that had been there originally. Pill size, color, number, etc. can be used for such comparison. The comparison can tell whether the pill administration has been correct. - In some embodiments, the vision
assistive device 100 can further register the compliance of the user with the medication administration schedule. For example, the user can be required to press a button to confirm the pill is taken. Or the user may be required to hold an open hand with a pill in it under a imaging unit of the visionassistive device 100 for capturing an image of it before taking the pill. The captured image may serve as evidence of the user having taken a pill. The captured image may even tell which pill was taken, upon image recognition by either a human or machine. The combination of the medical schedule calendar,control system 120, and the imaging unit in the visionassistive device 100 is essential for follow up to ensure compliance. Compliance (also adherence or capacitance) describes the degree to which a patient correctly follows medical schedule. Most commonly, it refers to medication or drug compliance, but it can also apply to other situations such as medical device use, self-directed exercises, or therapy sessions. - In some embodiments, the vision
assistive device 100 acts upon user's error. Thecontrol system 120 can be programmed so that in case the user makes an identifiable error (i.e. takes a wrong pill, has not taken it at all, etc.) the visionassistive device 100 can take appropriate measures, such as inform a caregiver and/or emergency service via email, SMS or automatic phone call, log the occurrence into a trouble list, or repeat the pill-time signal again later. - In some embodiments, the vision
assistive device 100 may be programmed to allow the user to play various games using the visionassistive device 100. For example, theimaging unit 44 can be used to capture images of crossword puzzles from paper, with user's hand-written scribbles on them. Then, upon image recognition and text recognition and processing, thecontrol system 120 can present the puzzle in a convenient format on thescreen 20. Such format can be displaying the question next to the corresponding column/row of the wanted word on thescreen 20. To get this convenient display, the user can point at either the puzzle question or the column/row of interest with a pen or finger. Thecontrol system 120 can identify such pointing by means of image recognition software, and display the puzzle question next to the corresponding column/row. - Other uses of recognition and processing of printed text captured by the
imaging unit 44 can be employed in the visionassistive device 100 of the present disclosure. For example, displaying magnified text on thescreen 20 can be useful for low vision users, making reading easier. The magnified display can be either in video mode (CCTV), showing the captured image as is, or upon OCR (optical character recognition) in an artificial font. Upon OCR and further upon text-to-speech conversion, the visionassistive device 100 can read aloud what the printed text says. Such speech output can be useful for blind users, making the vision assistive device 100 a reading machine. - In various embodiments, the vision
assistive device 100 of the present disclosure can be equipped with various sensors other than an imaging unit. Exemplary sensors include, but are not limited to, sensors for monitoring physiological data, such as blood pressure, heart rate, glucose level in blood, body temperature, etc. Additional sensors may include a microphone, smell sensor, fire alarm sensor, smoke detector, ambient temperature sensor, humidity sensor, atmospheric pressure sensor, carbon monoxide sensor, etc. The data collected by the sensors can be outputted on thescreen 20. In some embodiments, audio output can also be used. Medication administration schedule stored in thecontrol system 120 can be displayed or read aloud too. Switching to either of these modes of sensing and outputting or to a combination thereof can be done by means of pulling out, pushing in, sliding, or turning a large unit of control as described. - For example, a carbon monoxide sensor can be built-in within or connected to the vision assistive device 100 (by wire or wirelessly). When carbon monoxide level in the air exceeds a predetermined level, the vision
assistive device 100 can alarm the user with audio and/or visual messages. Such messages can reach other rooms wirelessly. The visionassistive device 100 can send such messages by email or/and phone to designated recipients such as relatives, neighbors, or/and fire department. Turning such a carbon monoxide sensor (or its connection to the vision assistive device 100) on and off can be done by means of pulling out, pushing in, sliding, or turning a large unit of control as described. A carbon monoxide sensor is just one example. Other sensors, such as blood pressure, heart rate, glucose level in blood, body temperature, etc., can be used similarly and so can their alarm messages. - In some embodiments, the user can control the vision
assistive device 100 by gestures. The printed text converted into magnified text on a monitor (for example as a scrolling line), or into speech, is intended for user's consumption. In the process of such output consumption, the user may wish to have control over the flow of the output text or speech. Specifically, such control may involve giving commands similar to what is called in other consumer players “Stop”, “Play”, “Fast-Forward” and “Rewind” commands. Commands such as “Zoom In”, “Zoom Out” can also be given by gestures, even though they may not be common in other consumer players. When such commands are to be given, the imaging unit is usually in video mode, yet is not monitoring turning pages in book-scanning setting. Thus, the imaging unit can be used to sense a specific motion or an image that signals to the algorithm that the corresponding command should be executed. For example, moving a hand in a specific direction under the imaging unit can signal one of the above commands. Moving a hand in a different direction under the imaging unit can signal a different command. In another example, the field of view of the imaging unit can be arranged to have a horizontal arrow that can be rotated by the user around a vertical axis. The image-processing algorithm can be pre-programmed to sense the motion and/or direction of the arrow. Such a motion can be detected and a change in the direction of the arrow can be identified as a signal. Here we call such a signal a “gesture”. A common software algorithm for the identification of the direction of motion, known as “Optical Flow” algorithm, can be utilized for such gesture recognition. - The interpretation of a gesture can be pre-programmed to depend on the current state of the output flow. For example, gesture interpretation can differ between the states in which 1) the text is being read out (in speech) to the user, 2) the text reading has been stopped, and 3) magnified text is being displayed. For example the gesture of moving a hand from right to left is interpreted as the “Stop” (aka “Pause”) command if the output text or speech is flowing. Yet, the same gesture can be interpreted as “Resume” (aka “Play”) if the flow has already stopped.
- Moving a hand in other manners can signal additional commands. For example, moving a hand back and forth (e.g. right and left), repeatedly, can signify a command, and repeating this movement a preset number of times within a preset time-frame can signify various additional commands. Gestures can also be interpreted as commands in modes other than output flow consumption. For example, in Video Mode, a gesture can give a command to change optical zoom or digital magnification. For this purpose, it is desirable to distinguish motion of a hand from other motion, such as motion of printed matter under the imaging unit. Optionally, the software that processes the video stream can recognize shapes of human fingers or the palm of the hand. With this capability, the software can distinguish motion of the user's hands from motion of the printed matter.
- In some embodiments, the vision
assistive device 100 may include a motion detector to switch on and off various functions of the visionassistive device 100. For example, the vision assistive device may be used for scanning of a book. During scanning of a book, alternating time intervals of motion and no motion can convey the process of turning pages. Such time intervals of motion and no motion can be considered as gestures too, even if the motion direction is irrelevant for the interpretation of the gesture. For example, as a page of a book is being turned, motion is being detected by motion detector software in thecontrol system 120, while monitoring video stream via an imaging unit that faces printed matter. The detected motion may be either that of a hand or that of printed matter. The fact that the page has been turned over and is ready for photographing is detected by the motion detector as the subsequent absence of motion. In practice, if motion (as observed by the detector) has dropped and stayed below a preset level for a preset time interval, the software interprets the drop as the page having been turned over. This may trigger taking a picture (photographing, capturing a digital image, a shot) and signaling this event to the user. Before the next shot is taken, the detector should see enough motion again and then a drop in motion for a long enough period of time. In this mode (e.g., book scanning), motion in any direction is being monitored, unlike in specific hand gesture recognition during output consumption, where motion in different directions may mean different commands. - In some embodiments, once a photograph of the target is stored, its text characters, word sets and other features can be used as recognizable images. The
control system 120 can assign those features to their coordinates (position) in the field of view (FOV) of the imaging unit. In other words, the images corresponding to a specific coordinate range are not pre-defined a priori but rather assigned to their position after photographing the target. One use of these page features is giving commands by covering and uncovering these images, e.g. by hand. This may include hand gestures seen as time sequences of covered and uncovered images (and thus their positions). The stored page image can work for this purpose as long as the page remains under the imaging unit with no appreciable shift relative to its location when imaged. The presence of the page with no shift may be monitored. If the page with no shift is absent, the algorithm can search 1) for another page or 2) for the same page with a shift. In both cases the images seen by the imaging unit can serve as a new predefined recognizable image set as described above. For example, moving a hand over such a page can signify various commands depending on the direction of the movement. For another example, covering a specific portion of such a page with a hand can also signify a command. By way of another non-limiting example, covering a subset of images located around a specific corner of the FO, as viewed from the imaging unit, may signal a command that is different from a command signaled by covering a subset of images around a different corner of the FOV. Such covering can be achieved by placing printed matter, a hand, or other objects on a subset of images or above it. The resulting commands can include the “Stop”, “Play”, “Fast-Forward” and “Rewind” commands, as well as activating the motion-detector mode for scanning a book. - Time sequences of such covered and uncovered images can be pre-programmed to encode various commands. A large number of commands can be encoded by such time sequences. Moving a hand above the surface of images in a specific manner can signal commands by way of covering and uncovering the images in various order (sequences). For example, moving a hand back and forth (e.g. right and left) can signify a command. Repeating this movement a preset number of times within a preset time-frame can signify various additional commands. In such gesture recognition, the shape of a hand can be used to differentiate such hand gestures from movement of printed matter over the surface. Such shape can be indicated by the silhouette of the set of images covered at any single time. Also, image recognition algorithms can be used for the purpose of recognizing hands, fingers, etc.
- In some embodiments, a vision assistive device includes a housing having a base for positioning the housing on a surface; one or more imaging units disposed along a top portion of the housing, each imaging unit being angled downward so a central axis of the imaging unit forms an acute angle with the surface so the imaging unit captures a target image from a target object; and a control system accommodated within the housing, the control system being connected to the one or more imaging units to process the target image captured by each imaging unit and to output the target image to a user.
- In some embodiments, a vision assistive device includes a housing having a base for positioning the housing on a surface; one or more imaging units configured to capture a target image from a target object positioned substantially perpendicular to the housing; a screen integrated with the housing; a control system accommodated within the housing, the control system being connected to the imaging units to process data corresponding to the image captured by the imaging units and to output the target image to a user on the screen.
- In some embodiments, a method for providing vision assistance includes disposing a housing substantially perpendicular to a surface, the housing including one or more imaging units and a control system; capturing with the one or more imaging units a target image from a target object placed on the surface adjacent to the housing; and processing the captured target image and outputting the target image on a screen integrated with the housing.
- All patents, patent applications, and published references cited herein are hereby incorporated by reference in their entirety. It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations, merely set forth for a clear understanding of the principles of the disclosure. Many variations and modifications may be made to the above-described embodiment(s) without departing substantially from the spirit and principles of the disclosure. It will be appreciated that several of the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into many other different systems or applications. All such modifications and variations are intended to be included herein within the scope of this disclosure, as fall within the scope of the appended claims.
Claims (20)
1. A vision assistive device comprising:
a housing having a base for positioning the housing on a surface, an opening, a top portion, and a front wall, back wall, and side walls connecting the base to the top portion and together defining an inner volume of the housing;
a neck slidably positioned within the opening of the housing, the neck having an up-position and down-position;
multiple cameras positioned upon the neck such that the cameras are positioned within the inner volume of the housing with the neck in the down-position and the cameras are positioned out of the inner housing when in the up-position so as to capture a target image from a target object;
a control system accommodated within the inner volume of the housing, the control system being connected to the multiple cameras to process the target image.
2. The vision assistive device of claim 1 wherein the cameras are positioned substantially over an edge of the target object positioned on the surface.
3. The vision assistive device of claim 1 wherein two cameras are provided that are spaced apart from one another.
4. The vision assistive device of claim 3 wherein the two cameras are configured to capture overlapping portions of the target image.
5. The vision assistive device of claim 4 wherein the control system is programmed to combine the overlapping portions.
6. The vision assistive device of claim 1 wherein a central axis of the cameras forms an acute angle with a plane defined by the front wall of the housing.
7. The vision assistive device of claim 1 further comprising a screen located between the base and the top portion of the housing for outputting the target image to the user.
8. The vision assistive device of claim 7 further comprising one or more rollers for scrolling through the target image displayed on the screen and enlarging desired portions of the target image.
9. The vision assistive device of claim 1 further comprising one or more sensors selected from the group consisting of smoke detector, carbon monoxide detector, temperature sensor, humidity sensor, motion detection sensor and combinations thereof.
10. A vision assistive device comprising:
a housing having a base for positioning the housing on a surface;
a retractable arm positioned within the housing;
a camera positioned upon the arm and configured to capture a target image from a target object positioned substantially perpendicular to the housing;
a screen integrated with the housing; and
a control system accommodated within the housing, the control system being connected to the camera to process data corresponding to the image captured by the cameras and to output the target image to a user on the screen.
11. The vision assistive device of claim 10 wherein a plurality of cameras are included upon the arm.
12. The vision assistive device of claim 10 wherein the arm spaces the camera away from the housing.
13. The vision assistive device of claim 10 wherein a central axis of the camera forms an acute angle with a plane defined by the front wall of the housing.
14. The vision assistive device of a claim 10 further comprising one or more analog controls for controlling one or more functionalities of the vision assistive device, the one or more functionalities being selected from the group consisting of alarm clock, calendar, security surveillance, digital frame, games, medication reminders and combinations thereof.
15. The vision assistive device of claim 10 further comprising one or more rollers for scrolling through the target image displayed on the screen and enlarging desired portions of the target image.
16. A vision assistive device comprising:
a housing having a base for positioning the housing on a surface;
a camera disposed along an adjustable arm, the arm selectively adjusting the position of the camera so that it is angled downward so a central axis of the camera forms an acute angle with the surface so the camera captures a target image from a target object; and
a control system accommodated within the housing, the control system being connected to the camera to process the target image captured by each imaging unit.
17. The vision assistive device of claim 16 wherein the camera is a CMOS camera.
18. The vision assistive device of claim 16 wherein the camera unit is configured to capture multiple portions of the target image with different focus positions and the control system is configured to combine the multiple portions.
19. The vision assistive device of claim 16 comprising two cameras spaced apart from one another.
20. The vision assistive device of claim 19 wherein the cameras are configured to capture overlapping portions of the target image.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/268,927 US20170069228A1 (en) | 2012-05-24 | 2016-09-19 | Vision Assistive Devices and User Interfaces |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201261688909P | 2012-05-24 | 2012-05-24 | |
US13/829,727 US8681268B2 (en) | 2012-05-24 | 2013-03-14 | Vision assistive devices and user interfaces |
US14/169,736 US9449531B2 (en) | 2012-05-24 | 2014-01-31 | Vision assistive devices and user interfaces |
US15/268,927 US20170069228A1 (en) | 2012-05-24 | 2016-09-19 | Vision Assistive Devices and User Interfaces |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/169,736 Continuation US9449531B2 (en) | 2012-05-24 | 2014-01-31 | Vision assistive devices and user interfaces |
Publications (1)
Publication Number | Publication Date |
---|---|
US20170069228A1 true US20170069228A1 (en) | 2017-03-09 |
Family
ID=49621317
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/829,727 Active US8681268B2 (en) | 2012-05-24 | 2013-03-14 | Vision assistive devices and user interfaces |
US14/169,736 Active US9449531B2 (en) | 2012-05-24 | 2014-01-31 | Vision assistive devices and user interfaces |
US15/268,927 Abandoned US20170069228A1 (en) | 2012-05-24 | 2016-09-19 | Vision Assistive Devices and User Interfaces |
Family Applications Before (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/829,727 Active US8681268B2 (en) | 2012-05-24 | 2013-03-14 | Vision assistive devices and user interfaces |
US14/169,736 Active US9449531B2 (en) | 2012-05-24 | 2014-01-31 | Vision assistive devices and user interfaces |
Country Status (4)
Country | Link |
---|---|
US (3) | US8681268B2 (en) |
EP (2) | EP3585042A1 (en) |
ES (1) | ES2753123T3 (en) |
WO (1) | WO2013177380A1 (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10971033B2 (en) | 2019-02-07 | 2021-04-06 | Freedom Scientific, Inc. | Vision assistive device with extended depth of field |
US11083286B2 (en) | 2019-02-13 | 2021-08-10 | Association for the Blind and Visually Impaired | Stand and system for assisting visual perception |
US20230053499A1 (en) * | 2021-08-23 | 2023-02-23 | Fujifilm Business Innovation Corp. | Image reading device |
JP7354002B2 (en) | 2020-02-05 | 2023-10-02 | Sanei株式会社 | Faucet control device and automatic faucet using it |
Families Citing this family (118)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8681268B2 (en) * | 2012-05-24 | 2014-03-25 | Abisee, Inc. | Vision assistive devices and user interfaces |
US9191554B1 (en) * | 2012-11-14 | 2015-11-17 | Amazon Technologies, Inc. | Creating an electronic book using video-based input |
GB201321857D0 (en) * | 2013-12-10 | 2014-01-22 | Samsung Electronics Uk Ltd | Display apparatus and integrated scanner |
US20150220114A1 (en) * | 2014-02-06 | 2015-08-06 | Abisee, Inc. | Devices, Systems and User Interfaces for Simplifying Control of Portable Computers |
KR20150102489A (en) | 2014-02-28 | 2015-09-07 | 삼성전자주식회사 | Display apparatus |
USD735197S1 (en) * | 2014-03-26 | 2015-07-28 | Optoelectronics Co., Ltd. | Optical information reader |
US20150310767A1 (en) * | 2014-04-24 | 2015-10-29 | Omnivision Technologies, Inc. | Wireless Typoscope |
WO2016028930A1 (en) * | 2014-08-19 | 2016-02-25 | Freedom Scientific, Inc. | Portable desktop video magnifier camera |
JP6753846B2 (en) | 2014-10-21 | 2020-09-09 | サノフィ−アベンティス・ドイチュラント・ゲゼルシャフト・ミット・ベシュレンクテル・ハフツング | Recording dose data from drug injection devices using optical character recognition |
JP6822950B2 (en) | 2014-10-21 | 2021-01-27 | サノフィ−アベンティス・ドイチュラント・ゲゼルシャフト・ミット・ベシュレンクテル・ハフツング | Recording dose data from drug injection devices using optical character recognition |
USD852252S1 (en) * | 2014-11-24 | 2019-06-25 | Nanolive Sa | Microscope |
US20160314708A1 (en) * | 2015-04-21 | 2016-10-27 | Freedom Scientific, Inc. | Method and System for Converting Text to Speech |
DE102015209022B4 (en) * | 2015-05-18 | 2020-08-20 | LS Asset Management GmbH & Co. KG | Screen reader |
JP6575802B2 (en) * | 2015-06-01 | 2019-09-18 | 柳吉 津野 | Anti-turning support for book stand, placement stand for book stand and USB stand scanner |
JP6903679B2 (en) | 2016-02-26 | 2021-07-14 | フリーダム サイエンティフィック インコーポレイテッド | Video magnifier |
CN109428980B (en) | 2017-08-23 | 2020-02-14 | 京东方科技集团股份有限公司 | Display device |
US10546444B2 (en) | 2018-06-21 | 2020-01-28 | Capital One Services, Llc | Systems and methods for secure read-only authentication |
US10542036B1 (en) | 2018-10-02 | 2020-01-21 | Capital One Services, Llc | Systems and methods for signaling an attack on contactless cards |
US10489781B1 (en) | 2018-10-02 | 2019-11-26 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10592710B1 (en) | 2018-10-02 | 2020-03-17 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10771254B2 (en) | 2018-10-02 | 2020-09-08 | Capital One Services, Llc | Systems and methods for email-based card activation |
BR112021004710A2 (en) | 2018-10-02 | 2021-06-08 | Capital One Services, Llc | system and method for transmitting data |
KR20210066798A (en) | 2018-10-02 | 2021-06-07 | 캐피탈 원 서비시즈, 엘엘씨 | System and method for cryptographic authentication of contactless card |
SG11202103249VA (en) | 2018-10-02 | 2021-04-29 | Capital One Services Llc | Systems and methods for cryptographic authentication of contactless cards |
US10841091B2 (en) | 2018-10-02 | 2020-11-17 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
WO2020072413A1 (en) | 2018-10-02 | 2020-04-09 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
WO2020072552A1 (en) | 2018-10-02 | 2020-04-09 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US11210664B2 (en) | 2018-10-02 | 2021-12-28 | Capital One Services, Llc | Systems and methods for amplifying the strength of cryptographic algorithms |
US10582386B1 (en) | 2018-10-02 | 2020-03-03 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10771253B2 (en) | 2018-10-02 | 2020-09-08 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10511443B1 (en) | 2018-10-02 | 2019-12-17 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
AU2019351911A1 (en) | 2018-10-02 | 2021-02-25 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10607214B1 (en) | 2018-10-02 | 2020-03-31 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10949520B2 (en) | 2018-10-02 | 2021-03-16 | Capital One Services, Llc | Systems and methods for cross coupling risk analytics and one-time-passcodes |
US10579998B1 (en) | 2018-10-02 | 2020-03-03 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
AU2019355436A1 (en) | 2018-10-02 | 2021-04-15 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
CA3115084A1 (en) | 2018-10-02 | 2020-04-09 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10733645B2 (en) | 2018-10-02 | 2020-08-04 | Capital One Services, Llc | Systems and methods for establishing identity for order pick up |
WO2020072670A1 (en) | 2018-10-02 | 2020-04-09 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10505738B1 (en) | 2018-10-02 | 2019-12-10 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
WO2020072694A1 (en) | 2018-10-02 | 2020-04-09 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10565587B1 (en) | 2018-10-02 | 2020-02-18 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10909527B2 (en) | 2018-10-02 | 2021-02-02 | Capital One Services, Llc | Systems and methods for performing a reissue of a contactless card |
CA3108399A1 (en) | 2018-10-02 | 2020-04-09 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10581611B1 (en) | 2018-10-02 | 2020-03-03 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US10680824B2 (en) | 2018-10-02 | 2020-06-09 | Capital One Services, Llc | Systems and methods for inventory management using cryptographic authentication of contactless cards |
US10554411B1 (en) | 2018-10-02 | 2020-02-04 | Capital One Services, Llc | Systems and methods for cryptographic authentication of contactless cards |
US20200226581A1 (en) | 2019-01-11 | 2020-07-16 | Capital One Services, Llc | Systems and methods for touch screen interface interaction using a card overlay |
US11037136B2 (en) | 2019-01-24 | 2021-06-15 | Capital One Services, Llc | Tap to autofill card data |
US11120453B2 (en) | 2019-02-01 | 2021-09-14 | Capital One Services, Llc | Tap card to securely generate card data to copy to clipboard |
US10510074B1 (en) | 2019-02-01 | 2019-12-17 | Capital One Services, Llc | One-tap payment using a contactless card |
US10467622B1 (en) | 2019-02-01 | 2019-11-05 | Capital One Services, Llc | Using on-demand applications to generate virtual numbers for a contactless card to securely autofill forms |
US10425129B1 (en) | 2019-02-27 | 2019-09-24 | Capital One Services, Llc | Techniques to reduce power consumption in near field communication systems |
US10523708B1 (en) | 2019-03-18 | 2019-12-31 | Capital One Services, Llc | System and method for second factor authentication of customer support calls |
US10438437B1 (en) | 2019-03-20 | 2019-10-08 | Capital One Services, Llc | Tap to copy data to clipboard via NFC |
US10535062B1 (en) | 2019-03-20 | 2020-01-14 | Capital One Services, Llc | Using a contactless card to securely share personal data stored in a blockchain |
US10984416B2 (en) | 2019-03-20 | 2021-04-20 | Capital One Services, Llc | NFC mobile currency transfer |
US10643420B1 (en) | 2019-03-20 | 2020-05-05 | Capital One Services, Llc | Contextual tapping engine |
US10970712B2 (en) | 2019-03-21 | 2021-04-06 | Capital One Services, Llc | Delegated administration of permissions using a contactless card |
US10467445B1 (en) | 2019-03-28 | 2019-11-05 | Capital One Services, Llc | Devices and methods for contactless card alignment with a foldable mobile device |
US11521262B2 (en) | 2019-05-28 | 2022-12-06 | Capital One Services, Llc | NFC enhanced augmented reality information overlays |
US10516447B1 (en) | 2019-06-17 | 2019-12-24 | Capital One Services, Llc | Dynamic power levels in NFC card communications |
US11694187B2 (en) | 2019-07-03 | 2023-07-04 | Capital One Services, Llc | Constraining transactional capabilities for contactless cards |
US10871958B1 (en) | 2019-07-03 | 2020-12-22 | Capital One Services, Llc | Techniques to perform applet programming |
US11392933B2 (en) | 2019-07-03 | 2022-07-19 | Capital One Services, Llc | Systems and methods for providing online and hybridcard interactions |
US10713649B1 (en) | 2019-07-09 | 2020-07-14 | Capital One Services, Llc | System and method enabling mobile near-field communication to update display on a payment card |
US10885514B1 (en) | 2019-07-15 | 2021-01-05 | Capital One Services, Llc | System and method for using image data to trigger contactless card transactions |
US10498401B1 (en) | 2019-07-15 | 2019-12-03 | Capital One Services, Llc | System and method for guiding card positioning using phone sensors |
US11182771B2 (en) | 2019-07-17 | 2021-11-23 | Capital One Services, Llc | System for value loading onto in-vehicle device |
US10733601B1 (en) | 2019-07-17 | 2020-08-04 | Capital One Services, Llc | Body area network facilitated authentication or payment authorization |
US10832271B1 (en) | 2019-07-17 | 2020-11-10 | Capital One Services, Llc | Verified reviews using a contactless card |
US11521213B2 (en) | 2019-07-18 | 2022-12-06 | Capital One Services, Llc | Continuous authentication for digital services based on contactless card positioning |
US10506426B1 (en) | 2019-07-19 | 2019-12-10 | Capital One Services, Llc | Techniques for call authentication |
US10541995B1 (en) | 2019-07-23 | 2020-01-21 | Capital One Services, Llc | First factor contactless card authentication system and method |
AU2019469080A1 (en) | 2019-10-02 | 2022-04-21 | Capital One Services, Llc | Client device authentication using contactless legacy magnetic stripe data |
US10776775B1 (en) | 2019-10-14 | 2020-09-15 | Capital One Services, Llc | NFC enabled card for consumer accessibility |
US10733283B1 (en) | 2019-12-23 | 2020-08-04 | Capital One Services, Llc | Secure password generation and management using NFC and contactless smart cards |
US10862540B1 (en) | 2019-12-23 | 2020-12-08 | Capital One Services, Llc | Method for mapping NFC field strength and location on mobile devices |
US11113685B2 (en) | 2019-12-23 | 2021-09-07 | Capital One Services, Llc | Card issuing with restricted virtual numbers |
US10885410B1 (en) | 2019-12-23 | 2021-01-05 | Capital One Services, Llc | Generating barcodes utilizing cryptographic techniques |
US10657754B1 (en) | 2019-12-23 | 2020-05-19 | Capital One Services, Llc | Contactless card and personal identification system |
US11615395B2 (en) | 2019-12-23 | 2023-03-28 | Capital One Services, Llc | Authentication for third party digital wallet provisioning |
US11651361B2 (en) | 2019-12-23 | 2023-05-16 | Capital One Services, Llc | Secure authentication based on passport data stored in a contactless card |
US10853795B1 (en) | 2019-12-24 | 2020-12-01 | Capital One Services, Llc | Secure authentication based on identity data stored in a contactless card |
US10664941B1 (en) | 2019-12-24 | 2020-05-26 | Capital One Services, Llc | Steganographic image encoding of biometric template information on a card |
US11200563B2 (en) | 2019-12-24 | 2021-12-14 | Capital One Services, Llc | Account registration using a contactless card |
US10757574B1 (en) | 2019-12-26 | 2020-08-25 | Capital One Services, Llc | Multi-factor authentication providing a credential via a contactless card for secure messaging |
US10909544B1 (en) | 2019-12-26 | 2021-02-02 | Capital One Services, Llc | Accessing and utilizing multiple loyalty point accounts |
US11038688B1 (en) | 2019-12-30 | 2021-06-15 | Capital One Services, Llc | Techniques to control applets for contactless cards |
US11455620B2 (en) | 2019-12-31 | 2022-09-27 | Capital One Services, Llc | Tapping a contactless card to a computing device to provision a virtual number |
US10860914B1 (en) | 2019-12-31 | 2020-12-08 | Capital One Services, Llc | Contactless card and method of assembly |
US11210656B2 (en) | 2020-04-13 | 2021-12-28 | Capital One Services, Llc | Determining specific terms for contactless card activation |
US10861006B1 (en) | 2020-04-30 | 2020-12-08 | Capital One Services, Llc | Systems and methods for data access control using a short-range transceiver |
US11030339B1 (en) | 2020-04-30 | 2021-06-08 | Capital One Services, Llc | Systems and methods for data access control of personal user data using a short-range transceiver |
US11823175B2 (en) | 2020-04-30 | 2023-11-21 | Capital One Services, Llc | Intelligent card unlock |
US11222342B2 (en) | 2020-04-30 | 2022-01-11 | Capital One Services, Llc | Accurate images in graphical user interfaces to enable data transfer |
US10915888B1 (en) | 2020-04-30 | 2021-02-09 | Capital One Services, Llc | Contactless card with multiple rotating security keys |
US10963865B1 (en) | 2020-05-12 | 2021-03-30 | Capital One Services, Llc | Augmented reality card activation experience |
US11063979B1 (en) | 2020-05-18 | 2021-07-13 | Capital One Services, Llc | Enabling communications between applications in a mobile operating system |
US11100511B1 (en) | 2020-05-18 | 2021-08-24 | Capital One Services, Llc | Application-based point of sale system in mobile operating systems |
US11062098B1 (en) | 2020-08-11 | 2021-07-13 | Capital One Services, Llc | Augmented reality information display and interaction via NFC based authentication |
USD961593S1 (en) * | 2020-08-26 | 2022-08-23 | Hand Held Products, Inc. | Barcode scanner |
US11165586B1 (en) | 2020-10-30 | 2021-11-02 | Capital One Services, Llc | Call center web-based authentication using a contactless card |
US11482312B2 (en) | 2020-10-30 | 2022-10-25 | Capital One Services, Llc | Secure verification of medical status using a contactless card |
US11373169B2 (en) | 2020-11-03 | 2022-06-28 | Capital One Services, Llc | Web-based activation of contactless cards |
US11216799B1 (en) | 2021-01-04 | 2022-01-04 | Capital One Services, Llc | Secure generation of one-time passcodes using a contactless card |
US11682012B2 (en) | 2021-01-27 | 2023-06-20 | Capital One Services, Llc | Contactless delivery systems and methods |
US11792001B2 (en) | 2021-01-28 | 2023-10-17 | Capital One Services, Llc | Systems and methods for secure reprovisioning |
US11687930B2 (en) | 2021-01-28 | 2023-06-27 | Capital One Services, Llc | Systems and methods for authentication of access tokens |
US11562358B2 (en) | 2021-01-28 | 2023-01-24 | Capital One Services, Llc | Systems and methods for near field contactless card communication and cryptographic authentication |
US11438329B2 (en) | 2021-01-29 | 2022-09-06 | Capital One Services, Llc | Systems and methods for authenticated peer-to-peer data transfer using resource locators |
US11777933B2 (en) | 2021-02-03 | 2023-10-03 | Capital One Services, Llc | URL-based authentication for payment cards |
US11637826B2 (en) | 2021-02-24 | 2023-04-25 | Capital One Services, Llc | Establishing authentication persistence |
US11245438B1 (en) | 2021-03-26 | 2022-02-08 | Capital One Services, Llc | Network-enabled smart apparatus and systems and methods for activating and provisioning same |
US11961089B2 (en) | 2021-04-20 | 2024-04-16 | Capital One Services, Llc | On-demand applications to extend web services |
US11935035B2 (en) | 2021-04-20 | 2024-03-19 | Capital One Services, Llc | Techniques to utilize resource locators by a contactless card to perform a sequence of operations |
US11902442B2 (en) | 2021-04-22 | 2024-02-13 | Capital One Services, Llc | Secure management of accounts on display devices using a contactless card |
US11354555B1 (en) | 2021-05-04 | 2022-06-07 | Capital One Services, Llc | Methods, mediums, and systems for applying a display to a transaction card |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040100575A1 (en) * | 2002-11-22 | 2004-05-27 | Thomas Malzbender | Retractable camera apparatus |
US20060257138A1 (en) * | 2005-05-10 | 2006-11-16 | Fromm Wayne G | Apparatus for supporting a camera |
US20070188626A1 (en) * | 2003-03-20 | 2007-08-16 | Squilla John R | Producing enhanced photographic products from images captured at known events |
US20070253703A1 (en) * | 2006-05-01 | 2007-11-01 | Quanta Computer Inc. | Built-in webcam |
US20080151056A1 (en) * | 2006-12-20 | 2008-06-26 | Chukwu Ahamefula | Automatic camera with inbuilt smoke and motion detectors that can send picture/pictures with audio or text |
US20080186287A1 (en) * | 2007-02-05 | 2008-08-07 | Nokia Corporation | User input device |
US20100215354A1 (en) * | 2009-02-24 | 2010-08-26 | Nikon Corporation | Imaging apparatus and image synthesis method |
US20110128825A1 (en) * | 2009-12-02 | 2011-06-02 | NL Giken Incorporated | Image Visualizing Device |
US20120154543A1 (en) * | 2010-12-15 | 2012-06-21 | Seiko Epson Corporation | Document camera, method for controlling document camera, program, and display processing system |
US20130215322A1 (en) * | 2012-02-20 | 2013-08-22 | Ken-A-Vision Manufacturing Company, Inc. | Document camera with automatically switched operating parameters |
Family Cites Families (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
USD254868S (en) | 1977-07-28 | 1980-04-29 | Hoadley Howard W | X-ray film viewer |
USD270277S (en) | 1982-06-28 | 1983-08-23 | Bethesda Research Laboratories, Inc. | Autoradiogram reader |
US4928170A (en) | 1988-06-21 | 1990-05-22 | Visualtek, Inc. | Automatic focus control for an image magnification system |
US4888195A (en) * | 1988-07-26 | 1989-12-19 | Nabisco Brands, Inc. | Ether bridged polyesters and food compositions containing ether bridged polyesters |
US5633674A (en) | 1994-12-12 | 1997-05-27 | Vision Technology, Inc. | Hand-held reading device for the visually impaired |
US6115482A (en) * | 1996-02-13 | 2000-09-05 | Ascent Technology, Inc. | Voice-output reading system with gesture-based navigation |
JPH10262147A (en) * | 1997-03-18 | 1998-09-29 | Minolta Co Ltd | Image reading device |
US6731326B1 (en) * | 1999-04-06 | 2004-05-04 | Innoventions, Inc. | Low vision panning and zooming device |
USD430588S (en) | 1999-08-04 | 2000-09-05 | Telesensory Corporation | Video magnifier for assisting persons with vision impairment |
US6791600B1 (en) * | 1999-08-11 | 2004-09-14 | Telesensory Corporation | Video system with dual mode imaging |
US6734913B1 (en) * | 1999-10-28 | 2004-05-11 | Hewlett-Packard Development Company, L.P. | Method of automatically adjusting exposure in a shutterless digital camera |
TW466415B (en) * | 2000-08-28 | 2001-12-01 | Compal Electronics Inc | Hand-held device with zooming display function |
US7075513B2 (en) * | 2001-09-04 | 2006-07-11 | Nokia Corporation | Zooming and panning content on a display screen |
US20040036663A1 (en) * | 2002-03-29 | 2004-02-26 | Bevers David R. | System and method for an image reader with electronic travel |
US6965862B2 (en) | 2002-04-11 | 2005-11-15 | Carroll King Schuller | Reading machine |
JP3941799B2 (en) * | 2003-08-22 | 2007-07-04 | カシオ計算機株式会社 | Camera stand |
US7805307B2 (en) | 2003-09-30 | 2010-09-28 | Sharp Laboratories Of America, Inc. | Text to speech conversion system |
US7840033B2 (en) | 2004-04-02 | 2010-11-23 | K-Nfb Reading Technology, Inc. | Text stitching from multiple images |
WO2005101349A1 (en) * | 2004-04-13 | 2005-10-27 | Pulse Data International Limited | Image magnifier for the visually impaired |
JP2005318023A (en) | 2004-04-27 | 2005-11-10 | Casio Comput Co Ltd | Electronic camera device, imaging method and program |
US8395664B2 (en) * | 2006-09-13 | 2013-03-12 | Smartvue Corp. | Wireless surveillance system and method for 3-D visualization and user-controlled analytics of captured data |
US8854441B2 (en) * | 2006-02-10 | 2014-10-07 | Freedom Scientific, Inc. | Electronic magnification device |
US7929013B2 (en) * | 2006-02-10 | 2011-04-19 | Freedom Scientific, Inc. | Desktop electronic magnifier |
US8854442B2 (en) * | 2006-02-10 | 2014-10-07 | Freedom Scientific, Inc. | Retainer for electronic magnification device |
US8594387B2 (en) * | 2007-04-23 | 2013-11-26 | Intel-Ge Care Innovations Llc | Text capture and presentation device |
KR100967128B1 (en) | 2007-11-23 | 2010-07-05 | 주식회사 힘스코리아 | Portable video magnifying apparatus |
US8113841B2 (en) | 2007-11-30 | 2012-02-14 | Beyo Gmbh | Reading device for blind or visually impaired persons |
EP2096614B1 (en) | 2007-11-30 | 2015-11-25 | beyo GmbH | Reading device for blind or visually impaired persons |
US20090244301A1 (en) * | 2008-04-01 | 2009-10-01 | Border John N | Controlling multiple-image capture |
TWM352703U (en) * | 2008-09-04 | 2009-03-11 | Quanta Comp Inc | Document snapshot device |
JP2010183420A (en) * | 2009-02-06 | 2010-08-19 | Elmo Co Ltd | Imaging device |
TWI387331B (en) * | 2009-04-22 | 2013-02-21 | Avermedia Information Inc | Document camera |
USD623214S1 (en) | 2009-10-20 | 2010-09-07 | Panasonic Corporation | Portable video magnifier |
JP5532992B2 (en) * | 2010-02-10 | 2014-06-25 | セイコーエプソン株式会社 | Document camera, document camera control method and program |
US20120001999A1 (en) * | 2010-07-01 | 2012-01-05 | Tandberg Telecom As | Apparatus and method for changing a camera configuration in response to switching between modes of operation |
US8749379B2 (en) * | 2012-01-19 | 2014-06-10 | Carefusion 303, Inc. | Portable medication dispensing containers |
US8681268B2 (en) | 2012-05-24 | 2014-03-25 | Abisee, Inc. | Vision assistive devices and user interfaces |
-
2013
- 2013-03-14 US US13/829,727 patent/US8681268B2/en active Active
- 2013-05-23 EP EP19190519.9A patent/EP3585042A1/en active Pending
- 2013-05-23 WO PCT/US2013/042391 patent/WO2013177380A1/en active Application Filing
- 2013-05-23 EP EP13794086.2A patent/EP2856746B1/en active Active
- 2013-05-23 ES ES13794086T patent/ES2753123T3/en active Active
-
2014
- 2014-01-31 US US14/169,736 patent/US9449531B2/en active Active
-
2016
- 2016-09-19 US US15/268,927 patent/US20170069228A1/en not_active Abandoned
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040100575A1 (en) * | 2002-11-22 | 2004-05-27 | Thomas Malzbender | Retractable camera apparatus |
US20070188626A1 (en) * | 2003-03-20 | 2007-08-16 | Squilla John R | Producing enhanced photographic products from images captured at known events |
US20060257138A1 (en) * | 2005-05-10 | 2006-11-16 | Fromm Wayne G | Apparatus for supporting a camera |
US20070253703A1 (en) * | 2006-05-01 | 2007-11-01 | Quanta Computer Inc. | Built-in webcam |
US20080151056A1 (en) * | 2006-12-20 | 2008-06-26 | Chukwu Ahamefula | Automatic camera with inbuilt smoke and motion detectors that can send picture/pictures with audio or text |
US20080186287A1 (en) * | 2007-02-05 | 2008-08-07 | Nokia Corporation | User input device |
US20100215354A1 (en) * | 2009-02-24 | 2010-08-26 | Nikon Corporation | Imaging apparatus and image synthesis method |
US20110128825A1 (en) * | 2009-12-02 | 2011-06-02 | NL Giken Incorporated | Image Visualizing Device |
US20120154543A1 (en) * | 2010-12-15 | 2012-06-21 | Seiko Epson Corporation | Document camera, method for controlling document camera, program, and display processing system |
US20130215322A1 (en) * | 2012-02-20 | 2013-08-22 | Ken-A-Vision Manufacturing Company, Inc. | Document camera with automatically switched operating parameters |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10971033B2 (en) | 2019-02-07 | 2021-04-06 | Freedom Scientific, Inc. | Vision assistive device with extended depth of field |
US11083286B2 (en) | 2019-02-13 | 2021-08-10 | Association for the Blind and Visually Impaired | Stand and system for assisting visual perception |
JP7354002B2 (en) | 2020-02-05 | 2023-10-02 | Sanei株式会社 | Faucet control device and automatic faucet using it |
US20230053499A1 (en) * | 2021-08-23 | 2023-02-23 | Fujifilm Business Innovation Corp. | Image reading device |
US11736636B2 (en) * | 2021-08-23 | 2023-08-22 | Fujifilm Business Innovation Corp. | Image reading device with display disposed between camera and placement section |
Also Published As
Publication number | Publication date |
---|---|
EP3585042A1 (en) | 2019-12-25 |
US9449531B2 (en) | 2016-09-20 |
US8681268B2 (en) | 2014-03-25 |
ES2753123T3 (en) | 2020-04-07 |
US20140146151A1 (en) | 2014-05-29 |
EP2856746B1 (en) | 2019-09-11 |
WO2013177380A1 (en) | 2013-11-28 |
EP2856746A4 (en) | 2016-06-01 |
US20130314593A1 (en) | 2013-11-28 |
EP2856746A1 (en) | 2015-04-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9449531B2 (en) | Vision assistive devices and user interfaces | |
US20160104051A1 (en) | Smartlight Interaction System | |
CN110546601B (en) | Information processing device, information processing method, and program | |
CN106341522A (en) | Mobile Terminal And Method For Controlling The Same | |
CN106416223A (en) | Imaging device and video generation method by imaging device | |
KR20140089132A (en) | Apparatus and method for taking a picture of portrait portable terminal having a camera and camera device | |
KR20160000741A (en) | Glass type terminal and control method thereof | |
CN105100590B (en) | Image display camera system, filming apparatus and display device | |
JP2006107048A (en) | Controller and control method associated with line-of-sight | |
JP7110443B2 (en) | Shooting method and shooting device, electronic equipment, storage medium | |
US20150220114A1 (en) | Devices, Systems and User Interfaces for Simplifying Control of Portable Computers | |
Toyama et al. | Wearable reading assist system: Augmented reality document combining document retrieval and eye tracking | |
JP5601142B2 (en) | Image display device, image display method, and program | |
JP6866467B2 (en) | Gesture recognition device, gesture recognition method, projector with gesture recognition device and video signal supply device | |
JP2017175580A (en) | Information display system and information providing terminal | |
US9148537B1 (en) | Facial cues as commands | |
JP2018077417A (en) | Display system, program, and display control method | |
WO2012008553A1 (en) | Robot system | |
JP6355146B1 (en) | Medical safety system | |
JP2012060547A (en) | Electronic mirror system | |
JP2016522929A (en) | Decoding method, apparatus, program, and recording medium | |
JP2011158959A (en) | Electronic manual display device | |
CN210955115U (en) | Desktop type auxiliary reading equipment | |
JP2007141018A (en) | Image acquisition method and image acquisition device | |
CN111698384B (en) | Image processing apparatus |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |