WO2016054860A1

WO2016054860A1 - Head wearing type vision auxiliary system for patient with vision disorder

Info

Publication number: WO2016054860A1
Application number: PCT/CN2014/093526
Authority: WO
Inventors: 杨小康; 翟广涛; 李铎; 闵雄阔
Original assignee: 上海交通大学
Priority date: 2014-10-10
Filing date: 2014-12-11
Publication date: 2016-04-14
Also published as: CN104306102A; CN104306102B

Abstract

A head wearing type vision auxiliary system for a patient with vision disorder comprises: a head wearing type glasses bracket (1), used for installing and connecting components of the system and helping a user to wear; a left eye display panel (3) and a right eye display panel (6), used for respectively displaying left and right eye images; a two-way image collection device (8), used for collecting images according to the visual angle of human eyes; a sensor (12, 13), used for detecting the external environment and the state of glasses; and a data processing center (9), used for overall control of the system, image collection, processing and projection display, reception and processing of data of the sensor and implementation of a related image processing algorithm. The head wearing type vision auxiliary system can collect reality scenes in real time, process the collected images, and display the images in a manner which is more acceptable to patients with vision disorder, and the objective of vision assistance is achieved.

Description

Head-mounted visual aid system for patients with visual impairment

Technical field

The present invention relates to the field of visual aid technology, and in particular to a head-mounted visual aid system for a visually impaired patient.

Background technique

Visual impairment, also known as "visual disability", "visual defect", "visual impairment" refers to some or all of the obstacles to the structure or function of the visual organs (including the eyeballs, the ocular nerves) and the cerebral optic nerve center caused by various reasons. It manifests as different degrees of visual impairment or narrowing of vision in both eyes, and it is difficult to achieve the work, study and other activities that ordinary people can do, which limits or hinders the normal effects that can be exerted according to their age, gender, social and cultural conditions.

According to estimates by the World Health Organization, there are 40 to 45 million blind people worldwide, and the low vision population is three times that of the blind, about 140 million people. Among the more than 83 million disabled people in China, there are 12.33 million visually handicapped persons, 5 million blind people and 7.1 million low-vision population. It is one of the countries with the largest number of blind people and low vision population in the world. How to solve their normal learning, work and life issues has become the focus of social attention.

Visual impairment is caused by innate or acquired causes of partial or total loss of function of the visual system, so that it is impossible to accurately identify external things. According to the relevant regulations, visual failure is less than 0.3, or visual field is within 20 °. According to the degree of obstacles, visual impairment can be divided into two categories: the visual acuity of the two eyes is less than 0.3 above 0.03 or the visual field is less than 20°; the visual acuity of the two eyes is less than 0.03 for total blindness. People with amblyopia can usually use vision for daily life, but there are certain obstacles in visual observation. Generally, optical aids or special magnification equipment are needed to observe things. But amblyopia itself is also very different: there is a lack of three-dimensionality, sensitivity to light, unrecognizable color, and too narrow a field of view. Therefore, the optical auxiliary equipment or special amplification equipment required is also very different. The blind population can't rely on vision for daily life at all, and usually can only take an alternative way to conduct daily life. For example, haptic aids, hearing aids, and other types of assistive devices are used in place of vision. It can be said that the visually impaired people need the most assistive technology and equipment. Most information barrier technologies and devices can be applied to people with visual impairments. Almost all visually impaired people can compensate for the quality of life and participate in social life through assistive devices. However, the second sample survey showed that only 7.97% of the visually impaired persons in China were assisted with services; 14.49% of the cities and 5.82% of the rural areas.

Visual impairment aids can be divided into two categories: visual and non-visual appliances:

Visual aids refer to devices or devices that can visually assist visually impaired persons to improve their visual ability. They are also classified into optical, non-optical, and electronic visual aids. Optical aids are more common, including hand-held, vertical magnifiers, glasses, monoculars, and binoculars. The optical aid is easy to use, easy to carry, low in price, and the main visual aid for low vision patients; however, it also has some limitations in use, such as low magnification, small reading range, long-term use. Causes poor posture, will be affected by the light source. Non-optical visual aid refers to improving visual effects by improving the surrounding environment. Such as: enlarged prints to change the size of the readings, high-intensity illuminators to increase contrast, high-contrast objects to facilitate the differentiation of local items. The electronic visual aid is an electronic video device that can read a document, a picture, an observed item, etc. through a photographic lens, and transmit the image to a screen for the user to browse. There are a wide variety of electronic vision aids, including: far-sighted, near-looking (zooming in), near-far, pocket-sized, portable, and so on. It is characterized by easy adjustment of magnification, brightness, contrast, etc., which can avoid glare; it has multiple display modes such as anti-white and half-color; it is ergonomically designed and can provide a large working space.

Non-visual aids refer to auxiliary devices that make full use of functions other than vision, such as hearing and touch, to compensate for the defects of visual functions and improve the quality of life of visually impaired people, including life, work, study, entertainment, etc. Every aspect of it. For example: blind mobile phones and telephones used for daily life, vocal clocks; blind canes and blind roads for travel; Braille for reading communication, dot display, audio books, various screen reading software, etc. Such aids are used more fully by blind patients.

In summary, the assistive technologies and equipment required by different visually impaired people are also different. The existing types of visual obstacle aids have the characteristics of single function and poor expandability, and do not conform to the principles of ease of use, application, simplicity, portability, and scalability.

Summary of the invention

In view of the deficiencies in the prior art, an object of the present invention is to provide a head-mounted visual aid system for a visually impaired patient, which can collect real-life scenes in real time, process the collected images and make the patients with visual impairment more Acceptable ways to improve the ability of visually impaired patients to obtain visual information to achieve the purpose of assisting vision.

To achieve the above object, the present invention provides a head-mounted visual aid system for a visually impaired patient, comprising:

Head-mounted eyeglass holder for mounting and connecting the various components of the system and for easy wearing by the user;

a display panel embedded in the frame of the headgear support for displaying an image;

An image capture device disposed on the headset holder;

a plurality of sensors for detecting external information and the state of the glasses and transmitting the signals to the data processing center;

a data processing center, wherein the data processing center is used for overall control of the system, including image acquisition, processing, and display of the display panel, and acceptance and processing of the sensor data; wherein the image processing is implemented by the image processing module, The image processing module collects images from the image acquisition device in real time, and processes the collected images. The image processing is processed by image processing and machine vision algorithms for different types of patients with visual impairment, and the processed images are visually impaired. The patient's more acceptable form is displayed in real time for visual aids.

Preferably, the display of the system is a binocular display or a monocular display; the image acquisition device is a dual channel acquisition or a single channel acquisition; if it is a single eye or a single channel, the display panel and the image acquisition device only need one way.

Further, if it is a binocular or a dual channel, the display panel and the image capturing device are two channels, that is, the display panel includes:

a left eye display panel embedded in the left eyeglass frame of the head mounted frame for displaying a left eye image;

a right eye display panel embedded in the right eyeglass frame of the head mounted glasses for displaying the right eye image;

The two-way image acquisition device is composed of two miniature cameras and is distributed on the left and right sides of the head-mounted glasses holder.

Preferably, the display mode of the system is a projection type or a non-projection type; if a projection type display is adopted, the system further comprises a projection device, and the projection device is projected on the display panel to form a virtual image at a certain distance in front of the human eye; The projection device is one way or two channels; if the non-projection display mode is adopted, the image is directly displayed on the screen of the display panel.

More preferably, the display panel of the non-projection display mode may be a high-definition liquid crystal display or an LED dot matrix display, and the selection of the display is adjusted according to the actual situation of the patient's patient.

More preferably, if the display panel is in a transmissive manner using a projection display mode, the image formed on the display panel is superimposed with the real scene to achieve a visual improvement effect.

Preferably, the display of the system may be either binocular or monocular; the image acquisition device may be dual channel acquisition or single channel acquisition.

Preferably, the two-way image acquisition device separately collects the real scenes seen by the left and right eyes at the perspective of the human eye, and performs real-time projection onto the left and right display panels after being processed by the data processing center.

Preferably, the data processing center image processing algorithm has different settings according to different functions required; different image processing algorithms correspond to different patients.

Compared with the prior art, the present invention has the following beneficial effects:

The invention is a wearable device, which has the advantages of simple and portable, easy to expand, and the like, and can make the visually impaired patient in the day Used in normal life; at the same time, the intelligent controller is adopted, in addition to the visual aid function, it can also have other functions of the smart device; in addition, the hardware disclosed by the present invention can be used as a visual obstacle assisting device platform for different types and different types. The degree of visual impairment can be designed in a targeted manner to achieve the purpose of assisting and enhancing vision. At the same time, for different types and degrees of visual impairment, a matching program can be designed specifically for the treatment and rehabilitation of the corresponding visual impairment.

DRAWINGS

Other features, objects, and advantages of the present invention will become apparent from the Detailed Description of Description

1 is a schematic structural view of a preferred embodiment of the present invention;

2 is a block diagram showing the working principle of a preferred embodiment of the present invention;

FIG. 3 is an effect diagram of a weak example of the embodiment of the present invention; wherein: (a) is a picture of a real scene captured by a camera, and (b) is a picture displayed by image processing (image enlargement).

In the figure: 1 is a head-mounted glasses holder, 2 is a left-eye glasses lens, 3 is a left-eye display panel, 4 is a left-eye projection device; 5 is a right-eye glasses lens, 6 is a right-eye display panel, and 7 is a right-eye display panel; Projection device, 8 is a two-way image acquisition device, 9 is a data processing center, 10 is a real scene picture collected by the two-way image acquisition device, 11 is a picture after being processed and projected, 12, 13 are sensors.

detailed description

The invention will now be described in detail in connection with specific embodiments. The following examples are intended to further understand the invention, but are not intended to limit the invention in any way. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the inventive concept. These are all within the scope of protection of the present invention.

FIG. 1 is a schematic structural diagram of a head-mounted visual aid system for a visually impaired patient according to a preferred embodiment of the present invention, in which: a head-mounted glasses holder 1, a left-eye glasses lens 2, and a left-eye display panel 3 a left eye projection device 4, a right eye lens 5, a right eye display panel 6, a right eye projection device 7, a two-way image acquisition device 8, a data processing center 9, and

sensors

12, 13 wherein:

The headgear support 1 is used for installing, connecting and fixing various components of the system, and is convenient for the user to wear;

The left eye lens 2 and the right eye lens 5 are used to respectively fix the left eye display panel 3 and the right eye display panel 6, and the left eye lens 2 and the right eye lens 5 are transparent. In addition to the user can see the vote In addition to the screen displayed, the actual scene can be seen through the lens;

The left-eye display panel 3 and the right-eye display panel 6 are embedded in a transparent spectacle lens at a certain angle and distance if the projection display mode is adopted, and the display panel is also transmissive. The virtual image displayed by the projection and the display scene seen through the lens are superimposed; if the non-projection display mode is adopted, the displayed image is a real image;

The left-eye projection device 4 and the right-eye projection device 7 cooperate with the left-eye display panel 3 and the right-eye display panel 6, respectively, and project images to the left-eye display panel 3 and the right-eye display panel 6, respectively. A virtual image is formed in front of the human eye;

The two-way image capturing device 8 is two cameras, and two cameras are respectively mounted on the left and right sides of the head-mounted glasses holder 1, and the real scenes seen by the left and right eyes can be separately collected in the perspective of the human eye in real time. It is transmitted to the data processing center 9, processed by the data processing center 9, and then projected to the left-eye display panel 3 and the right-eye display panel 6 in real time to form a virtual scene; in addition, since the two-way image acquisition device 8 is a person The two-way image is captured by the eye, and the collected left and right eye images are processed and projected to the left-eye display panel 3 and the right-eye display panel 6, respectively, so that the virtual scene seen by the human eye is a 3D scene;

The data processing center 9 is used for overall control of the system, image acquisition, processing and projection display; the data processing center 9 is used for overall control of the system and driving of each module.

In this embodiment, the data processing center 9 may be an existing processing data processing device on the market, such as a smart phone, a portable computer, or the like, or may be a self-developed data processing terminal, where the image processing algorithm is required according to the requirements. Different functions have different settings. For example, the data processing center 9 can have other functions of the smart device in addition to the image correction and calibration tasks for color blind patients.

In this embodiment, the data processing center 9 may be connected to the glasses through a data line, or may be directly connected to the glasses in a wireless manner.

In this embodiment, the

sensors

12 and 13 are used to detect external information and the state of the glasses and transmit the signals to the data processing center; and include various sensors such as an infrared sensor, an accelerometer, and a gyroscope, which are respectively collected. The data is passed to the data processing center 9, which is used by image processing and machine vision algorithms for analysis. The infrared sensor is used for collecting infrared signals for possible nighttime field of view detection or for obstacle distance detection; the accelerometer and the gyroscope respectively collect acceleration and angle of the glasses in various directions. By analyzing the data collected by these sensors, the data processing center can process the images to be displayed accordingly.

The main function of the system described in this embodiment is to provide a visual aid system for visually impaired patients, which not only can have different visual enhancement effects on different visual conditions, but also allows patients to rely on the aid of the instrument in their lives. Improve vision, and in the data processing center can be symptomatic to design different correction and treatment procedures, allowing patients to improve their visual ability during the practice.

In addition to the physical components described above, the system of the present embodiment further includes an image processing module designed for a visually impaired patient. The module collects images in real time from the two-way image acquisition device 8 and transmits them to the data processing center 9 to process the acquired images by using the image processing module. For different visually impaired patients, the corresponding image processing and machine vision algorithms are used to image the images. After processing, the more intuitive images of the more visually impaired patients are projected and displayed, and the visually impaired patient can view the processed images through the system, thereby achieving the purpose of assisting vision. specific:

For night blindness patients, increase the night vision of patients by increasing contrast sensitivity;

For patients with age-related macular degeneration, central serous retinopathy, cataract and other visual distortions, the patient will produce normal images through image deformation, such as displaying a grid map in the glasses, allowing the patient to adjust itself so that each The lines are all straight lines (in fact, the normal person sees the curved line after adjustment), and then the image captured by the camera produces the same deformation through image processing;

For red-green blind patients, let the computer change the red and green colors to the colors that the patient can distinguish, such as yellow and blue;

For low vision patients such as high myopia, let the computer enlarge the image for easy reading;

Increase the contrast and audible prompts for moving objects, so that patients are alert;

For visual disturbances caused by other conditions, the corresponding algorithm is used to improve or treat the patient's vision.

There are other patients and corresponding image processing methods that are not listed here. Changing the image processing algorithm does not affect the essence of the system. The image processing algorithm may use the prior art, or an algorithm that is innovative in order to adapt to the visual aid system.

As shown in Figure 2, which is the working principle block diagram of the system, the workflow includes the following steps:

First, image acquisition

The part is completed by two cameras of the two-way image acquisition device 8. The left and right cameras respectively collect images of the real scene from the perspective of the human eye, and the real scene image 10 collected by the camera is transmitted to the data processing in the form of a video stream. Center 9;

Second, image processing

This part is completed by the data processing center 9 of the system, and the data processing center 9 performs specific processing on the image collected from the two-way image capturing device 8 by using an associated image processing algorithm, such as: enlargement, enhancement, color correction, etc.; Image processing and machine vision related algorithms are needed. The system requires high performance requirements for related image processing algorithms, and it is required to achieve real-time requirements. Therefore, real-time performance should be considered when designing image processing algorithms. The problem is that the algorithm with too high complexity and unable to meet the real-time requirements cannot be adopted; if there are some more complicated algorithms, the corresponding system should be simplified or innovated accordingly, so that the algorithm can better cooperate with the system. The image processing algorithm may use the prior art, or an algorithm that is innovative in order to adapt to the visual aid system. The images collected by the two-way image acquisition device 8 are processed in real time and then transmitted to the left-eye display panel 3 and the right-eye display panel 6 respectively;

Third, the projection display (for transmissive display methods) or display using the display (for non-projection display),

The display mode of the system of the present invention may be a projection type or a non-projection type. If a projection display is used, wherein the left and right eye projection devices are respectively projected on the left and right display panels 3, 6, a virtual image is formed at a certain distance in front of the human eye; if a non-projection display mode is adopted, the real image is formed, but A convex lens is added between the glasses and the left and right display panels 3, 6 to make the image clearer. The display panel of the non-projection display can be a high-definition liquid crystal display or an LED dot matrix display, and the selection of the display can be adjusted according to the actual condition of the patient's patient. This does not affect the substance of the system. specific:

1) Projection display (for transmissive display method)

This portion is completed by the left and right

eye projection devices

4, 7 and the left and right eye display panels 3, 6, and the left and right

eye projection devices

4, 7 correct the color-blind image transmitted from the data processing center 9, and the left after calibration The right eye screen is respectively projected to the left and right eye display panels 3, 6, so that the screen seen by the user from the left and right eye display panels 3, 6 is an image processed by image processing and machine vision related algorithms;

2) Display using the display (for non-projected display mode)

The display is completed by the left and right display panels 3 and 6, and the left and right eye display panels 3 and 6 respectively display the color blind image corrected and the left and right eye images transmitted by the data processing center 9 to make the user The screens seen from the left and right display panels 3, 6 are images processed by image processing and machine vision related algorithms.

As shown in FIG. 3 , it is an effect diagram of the weak example, wherein: (a) is a real scene picture 10 collected by the two-way image acquisition device, and (b) is a related image processing (image enlargement) and then projected. The picture shown is 11.

Compared with the conventional visual impairment aid, the present invention is a wearable device, which has the advantages of simple portability, easy expansion, and the like, and can be used by visually impaired patients in daily life; in addition, the disclosed hardware can be used as a visual The obstacle assistive device platform can design a matching image processing program and a treatment program for different types and degrees of visual obstacles to achieve the purpose of assisting vision and treating diseases.

The specific embodiments of the present invention have been described above. It is to be understood that the invention is not limited to the specific embodiments described above, and various modifications and changes may be made by those skilled in the art without departing from the scope of the invention.

Claims

A head-mounted visual aid system for a visually impaired patient, comprising:

Head-mounted eyeglass holder for mounting and connecting the various components of the system and for easy wearing by the user;

a display panel embedded in the frame of the headgear support for displaying an image;

An image capture device disposed on the headset holder;

a plurality of sensors for detecting external information and the state of the glasses and transmitting the signals to the data processing center;

a data processing center, wherein the data processing center is used for overall control of the system, including image acquisition, processing, and display of the display panel, and acceptance and processing of the sensor data; wherein the image processing is implemented by the image processing module, The image processing module collects images from the image acquisition device in real time, and processes the collected images. The image processing is processed by image processing and machine vision algorithms for different types of patients with visual impairment, and the processed images are visually impaired. The patient's more acceptable form is displayed in real time for visual aids.
A head-mounted visual aid system for a visually impaired patient according to claim 1, wherein the display of the system is a binocular display or a monocular display; the image acquisition device is a dual channel acquisition or a single channel acquisition If it is a single eye or a single channel, the display panel and the image capturing device only need one way.
The head-mounted visual aid system for a visually impaired patient according to claim 2, wherein, if it is a binocular or a dual channel, the display panel and the image capturing device are two left and right paths, that is: The display panel includes:

a left eye display panel embedded in the left eyeglass frame of the head mounted frame for displaying a left eye image;

a right eye display panel embedded in the right eyeglass frame of the head mounted glasses for displaying the right eye image;

The two-way image acquisition device is composed of two miniature cameras and is distributed on the left and right sides of the head-mounted glasses holder.
The head-mounted visual aid system for a visually impaired patient according to claim 1, wherein the display manner of the system is a projection type or a non-projection type;

If a projection display is used, the system further includes a projection device, and the projection device is projected on the display panel to form a virtual image at a certain distance in front of the human eye; the projection device is one way or two ways;

If the non-projected display mode is used, the image is directly displayed on the screen of the display panel.
A head-mounted visual aid system for a visually impaired patient according to claim 4, wherein if a non-projection display mode is employed, a convex lens is further added between the eye and the display panel to enable imaging Clearer.
The head-mounted visual aid system for a visually impaired patient according to claim 5, wherein the display panel of the non-projection display mode is a high-definition liquid crystal display or an LED dot matrix display.
The head-mounted visual aid system for a visually impaired patient according to claim 4, wherein the display panel adopts a transmissive type, and an image formed on the display panel is superimposed with a real scene to achieve vision. Improved results.
A head-mounted visual aid system for a visually impaired patient according to claim 2, wherein the two-way image acquisition device separately collects real-life scenes seen by the left and right eyes from the perspective of the human eye. The data processing center processes the real-time projection onto the left and right display panels.
A head-mounted visual aid system for a visually impaired patient according to any one of claims 1-8, wherein the data processing center image processing algorithm has different settings according to different functions required. Different; different image processing algorithms correspond to different patients.
A head-mounted visual aid system for a visually impaired patient according to claim 9, wherein for night blindness patients, the night vision of the patient is increased by increasing contrast sensitivity;

For patients with age-related macular degeneration, central serous retinopathy, and cataract, these patients have normal images by image deformation;

For red-green blind patients, let the computer change red and green to the color that the patient can distinguish;

For low vision patients, let the computer enlarge the image for easy reading;

Increase the contrast and audible prompts for moving objects to alert the patient;

For visual disturbances caused by other conditions, the corresponding algorithm is used to improve or treat the patient's vision.