CN113687513A

CN113687513A - A regional visual field transfer system for people with visual field loss

Info

Publication number: CN113687513A
Application number: CN202110985822.XA
Authority: CN
Inventors: 吴铭; 王诗尊; 张闯; 陈恺鑫; 马占宇
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Beijing University of Posts and Telecommunications
Priority date: 2021-08-26
Filing date: 2021-08-26
Publication date: 2021-11-23

Abstract

The invention discloses an area vision transfer system for people with missing vision, comprising AR glasses, a computing unit and a controller, wherein the AR glasses include a front camera and an optical display element, which are respectively responsible for capturing pictures and displaying pictures, and the computing unit is responsible for Processing image data and logic processing, the controller is responsible for controlling the position, size, shape and transparency of the field of view of the area, and the AR glasses and the controller are electrically connected to the controller respectively. The front camera of the AR glasses of the present invention is used to capture the picture in front of the glasses in real time, and then the picture is transferred and superimposed to other positions in the field of view of the AR glasses. For visually impaired people with different visual field loss parts, the user can use the controller to flexibly adjust the position, size, shape and transparency of the target visual field area, and move and select the compensation position of the target visual field area in real time. The present invention is simple to implement and has remarkable effects, and can allow the visually impaired to see the complete world again without being misleading.

Description

Regional visual field transfer system for people with visual field loss

Technical Field

The invention relates to the technical field of computer vision, in particular to a regional visual field transfer system for people with visual field loss.

Background

The visually impaired person is called the visually impaired person for short, and in a broad sense, if an individual needs to see clearly through auxiliary devices such as glasses and magnifying glasses, the visually impaired person is called the visually impaired person. The visual impairment can be divided into the totally blind and the amblyopia, wherein the amblyopia is the most, the name of amblyopia means that the visual function is damaged or obstructed, so how to assist and enhance the visual ability of the visually impaired through the existing scientific technology is the key to solve the problem.

In order to solve the above problem, CN 105267013a discloses a head-mounted intelligent vision-impairment assisting system. The multifunctional glasses comprise a glasses end component and a functional end component, wherein the glasses end component comprises a camera, an image driving unit and an audio processor, and the camera is connected with the image driving unit; the functional end component comprises a communication module, a battery and charging management module, a central processing module and a key, and the image driving unit, the audio processor, the communication module, the battery and charging management module and the key are respectively connected with the central processing module; the central processing module comprises a fingertip detection module, a sensitive information detection module, an identification module and a voice processing module. The system is a wearable hardware system designed in a light weight mode, is low in power consumption and convenient to carry, realizes each core service module based on an embedded platform, comprises face recognition, object recognition, currency recognition and the like, and is simple, natural, intelligent, simple and easy to operate in interactive design.

However, the prior art does not consider the requirements of the people with local visual loss, i.e. the prior art does not design corresponding auxiliary functions for the people with local visual loss. Meanwhile, in the prior art, the visually impaired people receive information passively, and the visually impaired people can make judgment, so that the error rate is high. In addition, the device is complex to operate, large in size, inconvenient to carry, and exaggeratedly strange in shape, and does not consider the sensitive psychology of the visually impaired.

Local vision loss refers to loss of local detailed vision, and there are central vision loss and peripheral vision loss. Loss of central vision refers to loss of detailed vision. This feels like there is a lack of detail or a blurred spot in the central part of your field of view. As the lesion progresses, the blurred spot becomes darker or darker (as shown in fig. 1). This is commonly associated with macular degeneration and diabetic retinopathy. Peripheral vision loss, sometimes referred to as tunnel vision, is a perception that the eye is looking through a thin tube because it affects a wide angle field of view. The peripheral vision of the central vision is lost and only a narrow circular tunnel-like region remains in the field of view (as shown in fig. 2). Peripheral vision loss is commonly associated with glaucoma and retinitis pigmentosa.

Augmented Reality (AR) is a technology for calculating the position and angle of a camera image in real time and adding a corresponding image, and aims to overlap a virtual world on a screen in the real world and interact with the virtual world. In other words, when the real world is displayed on the screen through images, the AR technology adds a controllable programming layer between the real world and the user, thereby adding a new experience dimension. Through AR application, a user can interact more deeply with the real world, can perform operations in real time and obtain feedback, rather than just view information.

Aiming at the problem that the local picture of the visual field of the visually impaired people with local visual loss is speckled or invisible, the visual field picture is adjusted to be in a proper size through the controller based on the picture captured by the camera on the AR glasses, and the position of the visual field picture is adjusted to be in the area of the AR glasses picture which can be seen by the visually impaired people, so that the visually impaired people can see a complete world.

Disclosure of Invention

The invention aims to provide a regional visual field transfer system and an auxiliary method for people with visual field loss.

In order to achieve the above purpose, the invention provides the following technical scheme:

the invention firstly provides a regional visual field transfer system for people with missing visual fields, which comprises AR glasses, a computing unit and a controller, wherein the AR glasses comprise a front camera and an optical display element which are respectively responsible for capturing pictures and displaying the pictures, the computing unit is responsible for processing image data and logic processing, the controller is responsible for controlling the position, size, shape and transparency of the regional visual fields, and the AR glasses and the controller are respectively and electrically connected with the controller.

Furthermore, an embedded system is loaded on the computing unit to perform logic computation.

Further, the embedded system is an android system, Linux or raspberry pie

Furthermore, the optical display element of the AR glasses adopts a small prism type, a curved surface reflection type, an optical waveguide type or a pinhole imaging type optical imaging mode.

Further, the curved surface reflection type is a large curved surface or a small curved surface.

Further, the optical waveguide is a reflective optical waveguide, an etched grating waveguide or a holographic grating waveguide.

Further, the controller adopts a remote controller, a mobile phone, a touch panel or gesture recognition.

The invention also provides an auxiliary method of the regional visual field transfer system for people with visual field loss, which comprises the following steps:

s1, capturing a picture in front of the eyes by a front camera of the AR glasses, and sending the picture to a computing unit;

s2, the computing unit continuously monitors whether the controller sends an instruction or not while receiving the image;

s3, the controller sends an instruction for adjusting the size and the position of the image;

s4, the calculating unit adjusts the size and the position of the picture image captured by the camera according to the received instruction;

and S5, the computing unit sends the picture to an optical display element of the AR glasses for rendering and displaying.

Further, the controller instructions include an up-movement instruction, a down-movement instruction, a left-movement instruction, a right-movement instruction, and a zoom-in instruction, a zoom-out instruction.

Compared with the prior art, the invention has the beneficial effects that:

1. the regional visual field transfer system and the auxiliary method for the crowd with lost visual field, provided by the invention, have the advantages that the simple and effective function of regional visual field transfer is designed for the crowd with visual field loss, the visual field loss is compensated, the realization is simple, the effect is obvious, the visually impaired people can see the complete world positively, the misleading is avoided, and the visually impaired people can act according to the seen world.

2. The equipment is simple to operate, convenient to connect and strong in feedback.

3. The equipment has high definition and large field angle, and can bring better impression to the visually impaired people.

4. The equipment is convenient small and exquisite, is fit for carrying, has good wearing experience.

5. The equipment is beautiful and elegant, and fully takes care of the sensitive psychology of the visually impaired people. Drawings

In order to more clearly illustrate the embodiments of the present application or technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments described in the present invention, and other drawings can be obtained by those skilled in the art according to the drawings.

Fig. 1 is a schematic view of the field of vision of a person with central vision loss.

Fig. 2 is a view of a person with peripheral vision loss.

Fig. 3 is a schematic structural diagram of a regional visual field transfer system for people with visual field loss according to an embodiment of the present invention.

Fig. 4 is a flowchart of an auxiliary method of a regional visual field transfer system for people with visual field loss according to an embodiment of the present invention.

Fig. 5 is a schematic view of a visual field of a visually impaired person according to an embodiment of the present invention.

Fig. 6 is a diagram of a camera capturing frame according to an embodiment of the present invention.

Fig. 7 is a schematic view of a view field shift according to an embodiment of the present invention.

Detailed Description

For a better understanding of the present solution, the method of the present invention is described in detail below with reference to the accompanying drawings.

As shown in fig. 3, the present invention provides a system for transferring regional views to people with missing views, which includes AR glasses, a computing unit and a controller, wherein the AR glasses include a front camera and an optical display element, which are respectively responsible for capturing and displaying images, the computing unit is responsible for processing image data and logic processing, the controller is responsible for controlling the position, size, shape and transparency of the regional views, and the AR glasses and the controller are respectively electrically connected to the controller.

Specifically, the data line for the AR glasses is connected to the computing unit, the AR glasses are equivalent to a display, the computing unit is equivalent to a computer host, an Android system is mounted on the computing unit, and the function of visual field transfer is presented in the form of APP (Application). The APP is developed by using an SDK provided by an AR glasses manufacturer, a development platform is generally Unity, and the development platform is packaged and exported to an APK format (Android application package) after the development is completed, and then the APP is installed on a computing unit and can be used.

The optical display element of the AR glasses can adopt small prism type, curved surface reflection type, optical waveguide type or pinhole imaging type optical imaging modes. Wherein the curved surface reflection is a large curved surface or a small curved surface. The optical waveguide is a reflection optical waveguide, an etching grating waveguide or a holographic grating waveguide.

The small prism type AR glasses are projected on a small prism with a cut reflection surface to form an image, and a reflection mode (called "Birdbath") of a semi-transparent semi-reflective plane and a spherical surface is often adopted on a light propagation path. There are also products that use other mirrors for reflection, but prisms are generally the main ones. Typically, the field of view is about 20 degrees and the weight is about 50 grams. Big curved surface class AR glasses, good formation of image effect and super large angle of vision, the angle of vision can reach 110 degrees, brings very high experience of immersing, but is bulky, covers the position of forehead, and whole glasses are outstanding forward, are not convenient for daily wear for a long time. The small-curved-surface AR glasses are curved-surface reflection glasses which are close to the appearance of common glasses, and are combined by a prism or a coated reflection mirror, the principle is basically similar to that of a large curved surface, but the small-curved-surface AR glasses are benefited by an ultra-small display device, the vertical height is very compact, and the size and the thickness of the integrated sensing circuit can be the size of sunglasses. In general, the field angle can be 30-50 degrees, and the hollow small-curve module is light and only about 10 grams. The small free-form surface glasses are low in cost, good in imaging effect and suitable for wearing. Reflective waveguide AR glasses, the earliest AR solutions on the market that approximate ordinary glasses. The reflected light waveguide utilizes the total reflection of the projection light in the ultrathin glass to a specific position and is reflected into the eye for imaging. The field angle is about 30-50 degrees, and the weight can be within 100 grams. However, the process is complex (cutting, coating, pasting and polishing of glass), the production yield is low, and the cost is expensive. The diffraction light waveguide is totally reflected in the ultrathin glass by utilizing projection light, and the etched grating is contacted at a specific position to generate diffraction phenomenon and form an image by refraction. The field angle is about 30-50 degrees at present, and the weight of the whole machine is hundreds of grams. The holographic grating waveguide AR glasses use the total reflection of light in the ultrathin glass, meet the film of holographic material at a specific position, and perform reflection imaging. The general field angle is about 30-40 degrees, and the weight of the whole machine can be within 100 grams. The aperture combination mirrors use the optical waveguide and the micro-structured aperture array to image. The small hole microstructure can avoid the complex combination process of the common combined lens, can be realized on a very thin lens, and can have a larger refraction angle. The field angle is 80 degrees, and the weight can be within 100 grams. In summary, the present invention prefers small curve AR glasses. Specifically, this scheme equipment adopts Nreal Light, and the angle of vision: 52 ° FOV, screen resolution: 1080P; a camera: 500 ten thousand pixels, 2592 × 1944.

The controller can adopt a remote controller, a mobile phone, a touch panel or gesture recognition. The basic functions include moving, enlarging and reducing, adjusting the shape and transparency, and the like. For example, when the touch panel is slid up/down/left/right, the screen is moved up/down/left/right, the screen is enlarged by pressing a button on the remote controller, and the screen is enlarged by pressing a button on the remote controller. The shape and transparency are also adjusted by changing the shape by pressing a key, such as a double-click key, to decrease the transparency on the left side of the controller and to increase the transparency on the right side of the controller.

The invention also provides an auxiliary method of the regional visual field transfer system for people with visual field loss, as shown in fig. 4, the auxiliary method comprises the following steps:

s3, the controller sends instructions for adjusting the size, shape, transparency and position of the image;

The controller instructions include an up move instruction, a down move instruction, a left move instruction, a right move instruction, a zoom in instruction, a zoom out instruction, a change shape instruction, an increase transparency instruction, a decrease transparency instruction.

The visual field of the visually impaired (central vision loss) is shown in fig. 5, and by using the camera in front of the AR glasses of the present invention, the picture in front of the glasses is captured in real time (as shown in fig. 6), and then the picture is transferred and superimposed to other positions in the visual field of the AR glasses (as shown in fig. 7). Aiming at the visually impaired people with different vision loss parts, the user can flexibly adjust the position, size, shape and transparency of the target visual field area by using the controller, and the position compensated by the target visual field area is movably selected in real time. The invention has simple realization and obvious effect, really enables the visually impaired to see the complete world, has no misleading and enables the visually impaired to act according to the seen world.

The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: it is to be understood that modifications may be made to the technical solutions described in the foregoing embodiments, or equivalents may be substituted for some of the technical features thereof, but such modifications or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A regional field of vision transfer system for people with missing field of vision, characterized in that it comprises AR glasses, a computing unit and a controller, wherein the AR glasses include a front camera and an optical display element, which are respectively responsible for capturing images and displaying images, calculating The unit is responsible for processing image data and logic processing, the controller is responsible for controlling the position, size, shape and transparency of the field of view of the area, and the AR glasses and the controller are electrically connected to the controller respectively.

2 . The regional vision transfer system for people with missing vision according to claim 1 , wherein the computing unit is equipped with an embedded system to perform logical calculation. 3 .

3 . The regional visual field transfer system for people with visual loss according to claim 1 , wherein the embedded system is an Android system, a linux system or a Raspberry Pi. 4 .

4. The regional visual field transfer system for people with visual loss according to claim 1, wherein the optical display element of the AR glasses adopts a small prism type, a curved surface reflection type, an optical waveguide type or a pinhole imaging type optical imaging. Way.

5 . The regional visual field transfer system for people with missing visual field according to claim 4 , wherein the curved surface reflection class is a large curved surface or a small curved surface. 6 .

6 . The regional visual field transfer system for people with missing visual field according to claim 4 , wherein the optical waveguides are reflective optical waveguides, etched grating waveguides or holographic grating waveguides. 7 .

7 . The regional visual field transfer system for people with missing visual field according to claim 1 , wherein the controller adopts remote control, mobile phone, touch pad or gesture recognition. 8 .

8. The auxiliary method of the regional visual field transfer system for people with visual field deficiency according to any one of claims 1-7, characterized in that, comprising the following steps:

S1. The front camera of the AR glasses captures the picture in front of the eyes, and sends the picture to the computing unit;

S2. While receiving the image, the computing unit continuously monitors whether the controller sends an instruction;

S3. The controller sends an instruction to adjust the size and position of the image;

S4, the computing unit adjusts the size and position of the picture image captured by the camera according to the received instruction;

S5. The computing unit sends the picture to the optical display element of the AR glasses for rendering and display.

9 . The assistance method for the regional visual field transfer system for people with missing visual field according to claim 8 , wherein the controller instructions include an upward move instruction, a downward move instruction, a left move instruction, a right move instruction, and a zoom-in instruction. 10 . command, shrink command.