CN117297871A

CN117297871A - Blind glasses, blind glasses control method and blind glasses control device

Info

Publication number: CN117297871A
Application number: CN202311236830.XA
Authority: CN
Inventors: 周科霖; 李健; 陈明; 武卫东
Original assignee: Beijing Sinovoice Technology Co Ltd
Current assignee: Beijing Sinovoice Technology Co Ltd
Priority date: 2023-09-22
Filing date: 2023-09-22
Publication date: 2023-12-29

Abstract

The application provides blind glasses, a control method of the blind glasses and a control device of the blind glasses. The glasses for the blind comprise: the first image acquisition device is used for acquiring a first image of the blind person in the travelling direction; the controller is in communication connection with the first image acquisition equipment and is used for determining whether an obstacle exists in the travelling direction of the blind person according to the first image; the touch feedback device is in communication connection with the controller, and is used for performing touch feedback on the blind according to an obstacle avoidance instruction sent by the controller, wherein the obstacle avoidance instruction is an instruction sent by the controller when the controller determines that an obstacle exists in the travelling direction of the blind. The problem that the blind person navigation glasses in the prior art are poor in reminding effect for the blind person in a voice navigation mode is solved.

Description

Blind glasses, blind glasses control method and blind glasses control device

Technical Field

The application relates to the technical field of navigation glasses for blind persons, in particular to a pair of blind persons, a control method of the blind persons and a control device of the blind persons.

Background

Most of the current navigation glasses for the blind adopt a voice prompt mode to inform the blind how to act, but the voice command is relatively slow for the blind, and the situation that the voice command is touched when a complete obstacle is not broadcast can occur. Moreover, the obstacle avoidance and navigation modes of the blind through voice prompt are slower in response and limited by surrounding noise, and when the noise is too large, prompt voice can not be timely and clearly transmitted to the blind. Therefore, the current navigation glasses for the blind use voice prompt mode has poor reminding effect for the blind.

Disclosure of Invention

The main purpose of the application is to provide a pair of glasses for the blind, a control method of the pair of glasses for the blind and a control device of the pair of glasses for the blind, so as to at least solve the problem that the reminding effect of the navigation glasses for the blind is poor for the blind in the prior art by using a voice navigation mode.

In order to achieve the above object, according to one aspect of the present application, there is provided glasses for blind persons, comprising: the first image acquisition device is used for acquiring a first image of the blind person in the travelling direction; the controller is in communication connection with the first image acquisition equipment and is used for determining whether an obstacle exists in the travelling direction of the blind person according to the first image; the touch feedback device is in communication connection with the controller, and is used for performing touch feedback on the blind person according to an obstacle avoidance instruction sent by the controller, wherein the obstacle avoidance instruction is an instruction sent by the controller when the controller determines that the obstacle exists in the travelling direction of the blind person.

Optionally, the glasses for blind people further comprise: the second image acquisition device is in communication connection with the controller and is used for acquiring a second image of the eyeballs of the blind person, and the controller determines the visual direction of the blind person according to the second image.

Optionally, the glasses for blind people further comprise: the sound pickup device is used for collecting voice information of the blind person, wherein the voice information characterizes a grabbing target object; and the voice broadcasting equipment is in communication connection with the controller, and is used for carrying out voice feedback on the blind person according to the obstacle avoidance instruction sent by the controller.

Optionally, the haptic feedback device includes one or more of a pressure feedback device, a temperature feedback device, and a vibration feedback device.

According to another aspect of the present application, there is provided a method for controlling any one of the pair of glasses for blind, the method including: acquiring a first image, wherein the first image is an image of the blind person in the travelling direction; determining whether the obstacle exists in the first image by adopting a target detection algorithm, wherein the target detection algorithm comprises one or more of an R-CNN algorithm, a YOLO algorithm and an SSD algorithm; and under the condition that the obstacle exists in the first image, controlling the touch feedback device to output first prompt information, wherein the first prompt information is information of touch prompt and is used for prompting the blind person that the obstacle exists in the travelling direction.

Optionally, the pair of blind glasses further includes a second image acquisition device, where the second image acquisition device is communicatively connected to the controller, and the second image acquisition device is configured to acquire a second image of an eyeball of the blind, where the controller determines, according to the second image, a visual direction of the blind, and after acquiring the first image, the method further includes: acquiring the second image, wherein the second image is an image of the eyeball of the blind person; adopting an eyeball tracking technology to determine the visual direction of the eyeballs of the blind person; and determining a target traveling direction from the first image according to the visual direction, wherein the range of the traveling direction is larger than that of the target traveling direction.

Optionally, determining the eye-viewing direction of the eye of the blind person using eye-tracking technology includes: acquiring the center position of the pupil of the eyeball in the second image; and determining the direction in which the central position of the pupil is located as the visual direction.

Optionally, the pair of glasses for the blind person further includes a sound pickup device and a voice broadcasting device, where the sound pickup device is configured to collect voice information of the blind person, where the voice information characterizes a grabbing target object, and the voice broadcasting device is communicatively connected to the controller, where the voice broadcasting device is configured to perform voice feedback on the blind person according to the obstacle avoidance instruction sent by the controller, and after the first image is obtained, the method further includes: acquiring the voice information; determining a route for the blind person to grasp the target object according to the voice information; controlling the voice broadcasting equipment to output second prompt information, wherein the second prompt information is information of voice prompt, and the second prompt information is used for prompting the blind person of the direction of the target object; and controlling the touch feedback device to output third prompt information, wherein the third prompt information is information of touch prompts, and the third prompt information is used for prompting the blind person of the direction of the target object.

Optionally, after acquiring the first image, the method further comprises: determining whether a travel route in the travel direction of the blind person has gradient change according to the first image; controlling the voice broadcasting equipment to output fourth prompt information under the condition that the traveling route of the blind person in the traveling direction is determined to have gradient change, wherein the fourth prompt information is voice prompt information and is used for prompting the blind person to have gradient change in the traveling route in the traveling direction; and under the condition that the traveling route of the blind person in the traveling direction is determined to have gradient change, controlling the haptic feedback device to output fifth prompting information, wherein the fifth prompting information is information of haptic prompting, and the fifth prompting information is used for prompting the blind person to have gradient change in the traveling route in the traveling direction.

According to still another aspect of the present application, there is provided a control device for glasses for blind persons, the device comprising: a first acquisition unit configured to acquire a first image, wherein the first image is an image in a traveling direction of the blind person; a first determining unit configured to determine whether the obstacle exists in the first image by using an object detection algorithm, where the object detection algorithm includes one or more of an R-CNN algorithm, a YOLO algorithm, and an SSD algorithm; the first control unit is used for controlling the touch feedback device to output first prompt information under the condition that the obstacle exists in the first image, wherein the first prompt information is information of touch prompt, and the first prompt information is used for prompting the blind person that the obstacle exists in the travelling direction.

By the aid of the technical scheme, the touch feedback device is added in the glasses for the blind, when obstacles exist, the situation that the obstacles exist in front of the blind can be timely prompted through the touch feedback without being broadcasted by the equal voice broadcasting equipment, and the situation that the complete obstacles are touched when voice instructions are not broadcasted is avoided.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute an undue limitation to the application. In the drawings:

FIG. 1 shows a schematic diagram of the structure of the blind glasses of the present application;

fig. 2 is a block diagram showing a hardware configuration of a mobile terminal for performing a control method of glasses for blind in accordance with an embodiment of the present application;

fig. 3 is a schematic flow chart of a method for controlling glasses for blind people according to an embodiment of the present application;

FIG. 4 is a flow chart illustrating another method of controlling glasses for blind according to an embodiment of the present application;

fig. 5 shows a block diagram of a control device for glasses for blind person according to an embodiment of the present application.

Wherein the above figures include the following reference numerals:

102. a processor; 104. a memory; 106. a transmission device; 108. an input-output device; 10. a first image acquisition device; 20. a haptic feedback device; 30. and a second image acquisition device.

Detailed Description

It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In order to make the present application solution better understood by those skilled in the art, the following description will be made in detail and with reference to the accompanying drawings in the embodiments of the present application, it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, shall fall within the scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate in order to describe the embodiments of the present application described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

As described in the background art, the voice navigation mode of the blind person navigation glasses in the prior art has poor reminding effect for the blind person, and in order to solve the problems, the embodiment of the application provides the blind person glasses, the blind person glasses control method and the blind person glasses control device.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

The present solution provides a pair of glasses for blind person, as shown in fig. 1, comprising:

a first image capturing device 10 for capturing a first image of the blind person in the traveling direction;

the controller is in communication connection with the first image acquisition device and is used for determining whether an obstacle exists in the travelling direction of the blind person according to the first image;

and a tactile feedback device 20 communicatively connected to the controller, wherein the tactile feedback device is configured to perform tactile feedback on the blind person according to an obstacle avoidance command transmitted from the controller, the obstacle avoidance command being a command transmitted when the controller determines that the obstacle exists in the traveling direction of the blind person.

According to the embodiment, the touch feedback device is added into the glasses for the blind, when an obstacle exists, the situation that the obstacle exists in front of the blind can be timely prompted through the touch feedback without being completely broadcasted by the voice broadcasting equipment, and the situation that the complete obstacle is touched when a voice command is not broadcasted is avoided.

Specifically, the scheme adopts the mode of the head-wearing glasses, and the tactile feedback device arranged around the head of the blind person based on the head-wearing glasses can give the tactile feedback in the direction of the obstacle when the blind person is about to touch the obstacle in real time.

In particular, the tactile feedback may be a technique of applying a pressure signal to a human body through a piezo electric sheet or the like, thereby realizing tactile pressure feedback to the human body.

The existing blind person navigation equipment performs target positioning, mainly, objects such as obstacles on the blind person path are informed by shooting images of the blind person moving path and identifying and judging objects in the images based on machine vision, the acquired image information is more and overrules, the identification speed is affected, the accurate judgment of the target objects is not affected, and the positioning of the blind person's own visual direction is not performed, so that the identified objects and the blind person's own sensory targets cannot be unified accurately, the uncoordinated actions of the blind person and the prompts of the navigation equipment are likely to be caused, and in the specific implementation process, as shown in fig. 1, the blind person glasses further comprise a second image acquisition equipment 30 which is in communication connection with the controller and is used for acquiring a second image of eyeballs of the blind person, wherein the controller determines the visual direction of the blind person according to the second image.

In the scheme, the second image acquisition equipment can be added into the glasses for the blind, so that the second image of the eyeballs of the blind can be acquired, and the 'visual' direction of the blind can be determined, so that the consistency of subjective consciousness of the blind and equipment perception can be ensured according to the 'visual' direction, and the coordination of equipment prompt information on the action guidance of the blind is finally promoted.

Specifically, as shown in fig. 1, the pair of glasses for blind people in the present embodiment may include 4 first image capturing devices, where the first image capturing devices may be one or more of a camera, and a high-definition camera, and of course, the number of the first image capturing devices is not limited. The glasses for blind people can comprise 2 second image acquisition devices, wherein the second image acquisition devices can be one or more of cameras, cameras and high-definition cameras, and the number of the second image acquisition devices is not limited.

The touch feedback device is arranged at the surrounding position of the head of the blind person navigation glasses, the blind person head is surrounded by a circle of touch feedback device, and a group of devices are arranged at every 10 degrees of the horizontal direction angle to distinguish the horizontal direction of 360 degrees. The haptic feedback device comprises an upper haptic feedback device and a lower haptic feedback device in each direction, and the upper haptic feedback device and the lower haptic feedback device are used for distinguishing the upper haptic feedback device and the lower haptic feedback device. Thus, a total of 36 sets of 72 haptic feedback devices.

The direction and distance between the obstacle perceived by the equipment and the target object are transferred to the blind person in a touch prompt mode by adding a circle of touch feedback devices with 72 dimensions in 360-degree directions and up-down directions to the blind person head-mounted glasses equipment, so that the blind person is helped to improve the perception capability of the environment.

In a specific implementation process, the pair of blind glasses further comprises a sound pickup device and a voice broadcasting device, wherein the sound pickup device is used for collecting voice information of the blind, and the voice information characterizes a grabbing target object; the voice broadcasting device is in communication connection with the controller, wherein the voice broadcasting device is used for carrying out voice feedback on the blind person according to the obstacle avoidance instruction sent by the controller.

In the scheme, for the navigation glasses, interaction with the blind can be performed, voice of the blind can be picked up through the pickup equipment, then the controller determines a target object to be picked up by the blind, and further voice feedback is performed to prompt the direction of the target object of the blind.

In one implementation, the haptic feedback device includes one or more of a pressure feedback device, a temperature feedback device, and a vibration feedback device. Of course, the haptic feedback device is not limited to the above-described ones, but may include other haptic feedback devices, such as a current haptic feedback device.

In addition, can also add bluetooth headset as voice broadcast equipment in blind person's glasses, the bluetooth headset can be carried to the blind person like this, because bluetooth headset has the function of denoising, also can get rid of partial ambient noise.

The method embodiments provided in the embodiments of the present application may be performed in a mobile terminal, a computer terminal or similar computing device. Taking the operation on the mobile terminal as an example, fig. 2 is a block diagram of a hardware structure of the mobile terminal of a control method for glasses for blind in an embodiment of the present invention. As shown in fig. 2, the mobile terminal may include one or more (only one is shown in fig. 2) processors 102 (the processor 102 may include, but is not limited to, a microprocessor MCU or a processing device such as a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may further include a transmission device 106 for communication functions and an input-output device 108. It will be appreciated by those skilled in the art that the structure shown in fig. 2 is merely illustrative and not limiting of the structure of the mobile terminal described above. For example, the mobile terminal may also include more or fewer components than shown in fig. 2, or have a different configuration than shown in fig. 2.

The memory 104 may be used to store a computer program, for example, a software program of application software and a module, such as a computer program corresponding to a display method of device information in an embodiment of the present invention, and the processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, that is, to implement the above-described method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, which may be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, simply referred to as NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is configured to communicate with the internet wirelessly.

In the present embodiment, there is provided a control method of the glasses for blind person operating on a mobile terminal, a computer terminal or the like, it should be noted that the steps shown in the flowcharts of the drawings may be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in an order different from that herein.

Fig. 3 is a flow chart of a method for controlling glasses for blind according to an embodiment of the present application. As shown in fig. 3, the method comprises the steps of:

step S201, acquiring a first image, wherein the first image is an image in the travelling direction of the blind person;

specifically, the first image acquisition device is arranged in the pair of blind glasses, and the first image in the travelling direction of the blind can be acquired through the first image acquisition device.

Step S202, determining whether the obstacle exists in the first image by adopting an object detection algorithm, wherein the object detection algorithm comprises one or more of an R-CNN algorithm, a YOLO algorithm and an SSD algorithm;

specifically, there may be an obstacle in the traveling direction of the blind person, which affects the traveling of the blind person, and the blind person cannot see the obstacle, so that it is possible to automatically determine whether the obstacle exists in the traveling direction of the blind person by means of a machine.

And step 203, controlling the haptic feedback device to output a first prompt message when the obstacle exists in the first image, wherein the first prompt message is a message of a haptic prompt, and the first prompt message is used for prompting the blind person to have the obstacle in the travelling direction.

Specifically, under the condition that the traveling direction of the blind person is determined to have an obstacle, the blind person can be prompted in a touch prompting mode, the blind person can timely perceive the prompt through touch feedback, and the blind person can feel clear prompt.

Specifically, the first prompting information may be information of a pressure touch prompt, information of a temperature touch prompt, and information of a vibration touch prompt. Of course, the prompting can be performed in a mode of synchronously combining voice prompting.

Specifically, in the case of an obstacle in front, the pressure feedback device can be controlled to apply pressure, so that the blind can be prompted to have the obstacle in the traveling direction, and of course, for the distance of the obstacle, if the obstacle is closer, the pressure is larger, and if the obstacle is farther, the pressure is smaller.

Specifically, in the case of an obstacle in front, the temperature feedback device can be controlled to increase the temperature, so that the blind can be prompted to have the obstacle in the traveling direction, and of course, the closer the obstacle is, the higher the temperature is, and the farther the obstacle is, the lower the temperature is.

Specifically, in the case of an obstacle in front, the vibration feedback device can be controlled to vibrate, so that the blind can be prompted to have the obstacle in the traveling direction, and of course, if the obstacle is closer, the vibration frequency is faster, and if the obstacle is farther, the vibration frequency is faster and slower.

Compared with the current navigation equipment for the blind with only voice prompt, the navigation equipment for the blind can not accurately and rapidly give out specific azimuth and distance information of the obstacle and the target object. According to the scheme, 72 pieces of haptic feedback device equipment which encircle the head are added to the head-mounted navigation glasses equipment, the distance degree is distinguished in a pressure feedback strong and weak mode, the direction and distance of a barrier or a target object can be accurately and rapidly prompted for the blind under various noisy environments, and therefore the sensitivity and perceptibility of the blind in the sense of touch is further exerted.

The existing blind person navigation equipment performs target positioning, mainly, the objects such as obstacles on the blind person path are informed by shooting images of the blind person moving path and identifying and judging objects in the images based on machine vision, the acquired image information is multiple and overrules, firstly, the identification speed can be influenced, secondly, the accurate judgment of the target objects can be influenced, the positioning of the blind person's own visual' direction is not carried out, thus the identified objects and the blind person's own' sensory 'targets can not be unified accurately, the incompatibility of the blind person actions and the navigation equipment prompts is likely to be caused, the blind person navigation is not good enough, although the blind person can not see things, most of the blind person can not sense light due to organic lesions appearing on the blind person, but a large part of eyeballs of the blind person can control rotation according to the subjective intention and the target direction, in the specific implementation process, the blind person also comprises second image acquisition equipment, the second image acquisition equipment is communicated with the controller, the second image acquisition equipment is used for acquiring the second image of the blind person, the blind person's visual direction is determined according to the second image acquisition direction, and the method is further comprises the following steps: acquiring the second image, wherein the second image is an image of the eyeball of the blind person; an eyeball tracking technology is adopted to determine the visual direction of the eyeballs of the blind person; and determining a target traveling direction from the first image according to the visual direction, wherein the range of the traveling direction is larger than the range of the target traveling direction.

According to the scheme, through an eyeball tracking technology, semantic segmentation and directional semantic understanding can be carried out on a second image in the visual direction of the blind person, so that objects in the visual direction and angle which are subjectively wanted by the blind person can be accurately identified, and voice feedback and tactile feedback can be carried out subsequently, and further indiscriminate image understanding and excessive object identification noise can be avoided.

In addition, this embodiment may be adapted to blind persons with amblyopia or non-total blindness.

The blind person is tracked by eyeballs, and the oriented image understanding in the 'visual' direction of the eyeballs is aimed, so that the speed and the accuracy of the image understanding of the equipment are improved, the consistency of subjective consciousness and equipment perception of the blind person is ensured, and the coordination of the equipment prompt information on the action guidance of the blind person is finally improved.

Compared with the current blind auxiliary navigation scheme, the method for carrying out omnibearing image understanding on the acquired visual image. According to the method, the device and the system, the 'visual' direction of the blind person is subjected to directional image understanding by combining the eyeball tracking technology, so that the speed of processing image information by the system is improved, the subjective target of the blind person can be more accurately corresponding to the equipment identification result, and the consistency of the action purpose of the blind person and the equipment prompt is finally ensured.

Specifically, the main steps of semantic segmentation and directed semantic understanding of an image may include the following, data preparation steps: an image dataset with labels is collected, wherein the image labels include semantically segmented category labels and visual orientation labels. Model training: using deep learning methods, such as Convolutional Neural Networks (CNNs), training a model for semantic segmentation and directed semantic understanding, training can be performed using existing network architectures, such as U-Net, segNet, etc. A data preprocessing step: preprocessing the input image, including the operations of image resizing, normalization and the like, so as to input the image into a model for reasoning. An image segmentation step: inputting the preprocessed image into a trained model for semantic segmentation, and outputting class labels of each pixel point by the model to segment pixels of different classes. Directional semantic understanding step: the result of the semantic division is processed, information related to the visual direction is extracted, and the visual direction in the image can be determined by a method such as calculating probability distribution or angle regression of the visual direction. And a visual direction judging step: according to the result of directional semantic understanding, judging the visual direction in the image, and determining the visual direction by setting a threshold or selecting the category with the highest probability.

For example, in determining the direction of a pedestrian in an image, a set of labeled pedestrian image datasets may be collected, including semantic segmentation tags for pedestrians and direction tags for pedestrians. And performing model training by using the steps to obtain a model capable of performing semantic segmentation and oriented semantic understanding. Inputting an unknown pedestrian image, obtaining a semantic segmentation result after model reasoning, and judging whether the visual direction of the pedestrian is forward, left or right according to the result of directional semantic understanding.

For example, in determining the direction of a vehicle in a traffic scene, a set of annotated traffic scene image datasets may be collected, including semantic segmentation tags for vehicles and direction tags for vehicles. And performing model training by using the steps to obtain a model capable of performing semantic segmentation and oriented semantic understanding. Inputting an unknown traffic scene image, obtaining a semantic segmentation result after model reasoning, and judging whether the visual direction of the vehicle is forward, left or right according to the result of directional semantic understanding.

As can be seen from the above, the present solution mainly proposes two dimensional optimization solutions, in which, firstly, the haptic feedback device is combined to prompt the actions of the blind in time, and the ability of avoiding obstacle and positioning target of the blind in a noisy environment (when the voice prompt is affected to a certain extent) is improved through haptic perception; secondly, by carrying out eyeball tracking on the blind and aiming at the directional image understanding of the eyeball 'visual' direction, the problems of low recognition speed, untimely response, content recognition and chaotic prompt of the existing blind navigation equipment due to indiscriminate image understanding are solved.

In order to further accurately determine the visual direction of the blind person, so as to provide a more friendly navigation experience according to the visual direction, the visual direction of the eyeballs of the blind person is determined by adopting an eyeball tracking technology, which can be realized by the following steps: acquiring the center position of the pupil of the eyeball in the second image; and determining the direction in which the center position of the pupil is located as the visual direction.

In the scheme, more intelligent operation can be realized through the eyeball tracking technology, a plurality of directions are arranged around the blind person, the visual direction of the blind person can be determined through the eyeball tracking technology, and the position of the pupil in the common eyeball is changed according to the direction of expected advancing, so that the pupil in the second image can be identified, the direction of the central position of the pupil can be used as the target direction, the visual direction of the blind person can be accurately determined, and more friendly navigation experience can be provided subsequently.

The eye tracking technique is a technique for tracking and recording the movement locus and gaze point of the human eye in space. The human eye movement can be accurately detected and tracked in real time by using a high-speed camera and an infrared light source and combining a computer vision algorithm. By analyzing the eye movement pattern and gaze point, the attention and interests of the user may be obtained, thereby improving the product design and user experience.

The collected eyeball data can be input into an eyeball tracking algorithm, the visual direction of the eyeball is determined by analyzing the position and movement change of the eyeball, and the visual direction of a user can be identified in the left, right, up, down or specific directions according to the result of the eyeball tracking algorithm.

An image processing algorithm may be used to detect the eye region and locate the pupil. This may be achieved by thresholding, edge detection or other image processing techniques. Once the eye region is detected, a circular fitting algorithm or Hough circle transform, or the like, may be used to locate the center position of the pupil. These algorithms can find the best fit circle based on the shape and color characteristics of the pupil. During eye tracking, it is necessary to continuously update the center position of the pupil. Pupil tracking may be achieved using methods based on template matching, kalman filtering, or optical flow algorithms, among others. From the central position of the pupil, the gaze point or visual direction of the eye can be calculated. One common approach is to use geometric calculations or linear interpolation to map the center position of the pupil to the gaze point position on the screen coordinate system.

Specifically, the intensity of the haptic feedback can be adjusted according to the distance of the obstacle, and the intensity of the haptic feedback is weak and strong when the obstacle is far and near; when the target is navigated, the blind person is given tactile feedback at the opposite direction position according to the navigation path, and when the target is far or near, the tactile feedback is strong or weak.

In the case that the blind person wants to grasp the target object, the blind person can be prompted according to the combination of tactile feedback and sound feedback, and in some embodiments, the pair of glasses for the blind person further includes a sound pickup device and a voice broadcasting device, where the sound pickup device is used for collecting voice information of the blind person, the voice information characterizes grasping the target object, and the voice broadcasting device is in communication connection with the controller, where the voice broadcasting device is used for performing voice feedback on the blind person according to the obstacle avoidance instruction sent by the controller, and after obtaining the first image, the method further includes the following steps: acquiring the voice information; determining a route of the blind person for grabbing the target object according to the voice information; controlling the voice broadcasting device to output second prompt information, wherein the second prompt information is voice prompt information, and the second prompt information is used for prompting the direction of the target object for the blind; and controlling the touch feedback device to output third prompt information, wherein the third prompt information is information of a touch prompt, and the third prompt information is used for prompting the blind person of the direction of the target object.

In the scheme, the direction of the target object of the blind person can be prompted by combining the voice prompt and the touch prompt, so that the blind person can navigate under the condition that the blind person is to grasp the target object, and the friendliness of the navigation to the blind person is further improved.

Specifically, the voice information may be a voice that the blind person wants to grasp a target object, such as "grasp an apple in a house", or the voice information may be a voice that the blind person wants to go to a destination, such as "go to a mall".

Specifically, the second prompting information may be "the target object is on the left side", "the target object is in front of the hand please grasp", or the like, for prompting the direction of the target object for the blind person.

Specifically, the third prompting information may be information of a pressure touch prompt, information of a temperature touch prompt, or information of a vibration touch prompt.

Specifically, after the direction and distance between the blind person and the target object are determined, the pressure feedback device can be controlled to apply pressure, the pressure applied at this time can be different from the pressure applied by the obstacle in front, for example, the pressure applied by the obstacle is the pressure applied by the pressure touch feedback device in front of the brain, the pressure applied by the pressure touch feedback device in back of the brain when the target object is grabbed is the pressure applied by the pressure touch feedback device in back of the brain, which corresponds to the state of pushing the blind person away, if the pressure applied by the obstacle in front is the first pressure state, if the pressure applied by the obstacle in front is the second pressure state, and the pressure state can be the pressure, the pressure sense and the like. Thus, the blind person can be prompted to grasp the target object. Of course, the pressure is smaller if the target object is closer, and the pressure is larger if the target object is farther.

Specifically, after the direction and distance between the blind person and the target object are determined, the temperature feedback device can be controlled to change the temperature, the temperature increased at this time can be different from the temperature increased by the obstacle in front of the blind person, for example, the temperature increased by the temperature touch feedback device in front of the brain is the temperature increased by the temperature touch feedback device in front of the brain, and the temperature increased by the temperature touch feedback device in back of the brain when the target object is grabbed is equivalent to the state of pushing the blind person away, if the temperature is in a first temperature state when the obstacle is in front of the blind person, if the temperature is in a second temperature state when the target object is to be grabbed, the temperature state can be the temperature state of high or low, the temperature rise or the like. Thus, the blind person can be prompted to grasp the target object. Of course, the lower the temperature if the target object is closer, the higher the temperature if the target object is farther.

Specifically, after the directions and distances between the blind person and the target object are determined, the vibration feedback device can be controlled to vibrate, the vibration frequency of the vibration at this time can be different from the vibration frequency of the vibration of the obstacle in front of the target object, for example, the vibration of the obstacle is the vibration of the vibration touch feedback device in front of the brain, the vibration of the vibration touch feedback device in back of the brain when the target object is grabbed is equivalent to the vibration of the vibration feedback device in back of the brain, which is used for pushing the blind person, the vibration is the first vibration state if the obstacle is in front of the blind person, the vibration is the second vibration state if the target object is to be grabbed, and the vibration state can be the vibration frequency state. Thus, the blind person can be prompted to grasp the target object. Of course, the slower the vibration frequency if the target object is closer, the faster the vibration frequency if the target object is farther.

Through the scheme, the blind person can be timely prompted when the obstacle exists in front or the blind person wants to grasp the target object, the prompt information broadcasted by the voice broadcasting equipment is different, the prompt information output by the touch feedback device is different, and the blind person can distinguish the scene that the obstacle exists in front or the blind person wants to grasp the target object.

In addition, images of the hands of the blind and the target object can be acquired through the first image acquisition equipment, the distance between the hands of the blind and the target object is determined through a deep learning algorithm, and then second prompt information and third prompt information are generated according to the distance to prompt the blind to grasp the object.

The blind person can be prompted timely if the route has gradient change when the blind person walks, and in some embodiments, after the first image is acquired, the method further comprises the following steps: determining whether a travel route in the travel direction of the blind person has gradient change according to the first image; controlling the voice broadcasting device to output fourth prompt information under the condition that the traveling route of the blind person in the traveling direction is determined to have gradient change, wherein the fourth prompt information is voice prompt information and is used for prompting the blind person to have gradient change in the traveling route in the traveling direction; and controlling the haptic feedback device to output fifth prompt information when the slope change of the travel route of the blind person in the travel direction is determined, wherein the fifth prompt information is information of a haptic prompt, and the fifth prompt information is used for prompting the slope change of the travel route of the blind person in the travel direction.

According to the scheme, the route of the blind person can be prompted to have gradient change by combining the sound prompt and the touch prompt, so that the navigation prompt can be further accurately performed when the blind person walks, and the friendliness of navigation to the blind person is further improved.

Specifically, the fourth prompting message may be "up slope in front", "down slope in front", "need to go up stairs in front" or "need to go down stairs in front" and so on, and is used for prompting the blind people to have gradient change on the walking route.

Specifically, the fifth prompting information may be information of a pressure touch prompt, information of a temperature touch prompt, or information of a vibration touch prompt.

Specifically, as can be seen from fig. 1, the surrounding glasses have two belts, which are divided into upper and lower belts, and the pressure feedback device can be controlled to apply pressure when it is determined that the travel route of the blind person in the travel direction has a slope change. The pressure feedback device on the upper belt is applied with larger pressure when ascending, the pressure feedback device on the lower belt is applied with smaller pressure, or the pressure feedback device on the upper belt is applied with pressure after the pressure feedback device on the lower belt is applied with pressure. The pressure feedback device on the upper belt applies smaller pressure when descending a slope, the pressure feedback device on the lower belt applies larger pressure, or the pressure feedback device on the lower belt applies pressure after the pressure feedback device on the lower belt applies pressure. Thus, the gradient change of the blind can be prompted.

Specifically, as can be seen from fig. 1, the surrounding glasses have two belts, which are divided into an upper belt and a lower belt, and when it is determined that the travel route of the blind person in the travel direction has a gradient change, the temperature feedback device can be controlled to increase the temperature. When ascending, the temperature output by the temperature feedback device on the upper belt is higher, the temperature output by the temperature feedback device on the lower belt is lower, or the temperature feedback device on the upper belt outputs the temperature after the temperature feedback device on the lower belt. When descending a slope, the temperature output by the temperature feedback device on the lower belt is higher, the temperature output by the temperature feedback device on the upper belt is lower, or the temperature feedback device on the upper belt outputs the temperature after the temperature feedback device on the lower belt outputs the temperature. Thus, the gradient change of the blind can be prompted.

Specifically, as can be seen from fig. 1, the surrounding glasses have two belts, which are divided into upper and lower belts, and when it is determined that there is a slope change in the travel route of the blind person in the travel direction, the vibration feedback device can be controlled to vibrate. When ascending, the vibration frequency of the vibration feedback device on the upper belt is higher, the vibration frequency of the vibration feedback device on the lower belt is lower, or the vibration feedback device on the upper belt vibrates the vibration feedback device on the lower belt first and then vibrates. When descending a slope, the vibration frequency of the vibration feedback device on the lower belt is higher, the vibration frequency of the vibration feedback device on the upper belt is lower, or the vibration feedback device on the lower belt vibrates the vibration feedback device on the lower bag first and then. Thus, the gradient change of the blind can be prompted.

Of course, different haptic feedback may also be used for different application scenarios. For example, pressure haptic feedback is used when there is an obstacle in the traveling direction, temperature haptic feedback is used when the target object is to be grasped, and vibration haptic feedback is used when the gradient is changed. Of course, the method is not limited to the above situation, and different tactile feedback can be corresponding to different scenes, so that the blind can be reminded of distinguishing specific application scenes.

Specifically, the control method of the blind glasses can be executed by a controller in the blind glasses, and an instruction output by the controller is used for controlling the tactile feedback device.

In conclusion, the blind person is guided to avoid the obstacle and find the target object in a tactile feedback mode to achieve the purposes of avoiding the obstacle and finding the object more timely and accurately.

In order to enable those skilled in the art to more clearly understand the technical solutions of the present application, the implementation process of the control method of the blind glasses of the present application will be described in detail below with reference to specific embodiments.

The embodiment relates to a specific control method for glasses for blind persons, as shown in fig. 4, mainly comprising five modules: the system comprises 4 external photographing cameras, 2 internal photographing cameras, a video image information comprehensive analysis module, a path and target navigation module and a touch pressure feedback module.

The first step, a tactile pressure feedback device is deployed at the surrounding position of the head of the blind person navigation glasses, a circle of tactile feedback device is provided for the head of the blind person, and a group of devices are deployed at every 10 degrees of the horizontal direction angle to distinguish the horizontal direction of 360 degrees. The haptic device includes upper and lower haptic feedback devices in each direction for differentiating between the upper and lower directions. Thus, a total of 36 sets of 72 haptic feedback devices.

Secondly, arranging 4 outward shooting cameras at the front ends of four corners of the outer sides of the blind navigation glasses, and shooting video images of the visual directions of the blind; and arranging 2 inner shooting image heads at left and right positions on the inner side of the blind navigation glasses, wherein the inner shooting image heads are used for shooting blind eyeball video images.

And thirdly, transmitting the external environment images shot by the 4 external shooting cameras to a video image information comprehensive analysis module, and synchronously transmitting the blind eyeball images shot by the 2 internal shooting cameras to the video image information comprehensive analysis module.

And fourthly, comprehensively analyzing by combining the external shot image and the internal shot image by a video image information comprehensive analysis module, tracking the eyeballs of the blind person in the internal shot image to output eyeball 'visual' direction information, and further carrying out directional processing and analysis understanding of a demarcation range on the external shot image according to the eyeball 'visual' direction, thereby obtaining object information in the appointed direction for subsequent obstacle or target object judgment.

And fifthly, transmitting the object information of the blind 'visual' direction output by the video image information comprehensive analysis module to the path and target navigation module, judging whether the object is an obstacle on the blind action path or a target object which the blind wants to acquire by the module, and planning the action path (comprising the information of the direction in which the blind needs to travel, the obstacle which the blind needs to avoid, the target object and the like) for the blind based on the image information comprehensive analysis result.

And a sixth step of transmitting the movement route information generated by the path and the target navigation module to the tactile pressure feedback module, transmitting the movement route information to 72 tactile pressure feedback devices encircling the head of the blind person in a tactile pressure information mode according to the direction in which the blind person needs to travel and the position of the avoidance obstacle, so as to simulate the position of the obstacle or the position of the target object in advance, and transmitting the movement route information to the blind person in a tactile pressure prompting information mode (for example, for the obstacle, the tactile pressure devices in the same direction apply a certain degree of tactile pressure feedback to the blind person according to the direction in which the obstacle exists, the feedback intensity is different according to the distance of the obstacle, the pressure is similar to that of the blind person when the obstacle is far and near, and the pressure is similar to that of pushing the blind person to the specified direction when the target object is far and near, and for the target object, the tactile pressure device in the opposite direction applies a certain degree of tactile pressure feedback to the blind person according to the direction in which the target object exists, the feedback intensity is different from the distance and the pressure is similar to that the blind person is pushing to the specified direction.

Seventh, optionally, the tactile pressure feedback device can synchronously assist with voice prompt when the blind person is in tactile prompt, so that the accuracy of prompt information transmission is further ensured.

The embodiment of the application also provides a control device for the blind glasses, and the control device for the blind glasses can be used for executing the control method for the blind glasses. The device is used for realizing the above embodiments and preferred embodiments, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

The following describes a control device for glasses for blind provided in the embodiment of the present application.

Fig. 5 is a block diagram of a control device for glasses for blind according to an embodiment of the present application. As shown in fig. 5, the apparatus includes:

a first acquisition unit 100 for acquiring a first image, wherein the first image is an image in a traveling direction of the blind person;

A first determining unit 200 configured to determine whether the obstacle exists in the first image by using an object detection algorithm, where the object detection algorithm includes one or more of an R-CNN algorithm, a YOLO algorithm, and an SSD algorithm;

and a first control unit 300 for controlling the haptic feedback device to output a first prompt message when it is determined that the obstacle exists in the first image, wherein the first prompt message is a message of a haptic prompt, and the first prompt message is used for prompting the blind person to have the obstacle in the traveling direction.

The existing blind person navigation equipment performs target positioning, mainly, objects such as obstacles on the blind person path are informed by shooting images of the blind person movement path and identifying and judging objects in the images based on machine vision, the acquired image information is multiple and overrules, firstly, the identification speed can be influenced, secondly, the accurate judgment of the target objects can be influenced, and the positioning of the blind person's own visual direction is not carried out, so that the identified objects and the blind person's own sensory targets cannot be unified accurately, the incompatibility of the blind person actions and the navigation equipment prompts is likely to be caused, the blind person navigation is insufficient, the blind person can not see things, but most of the blind person can not sense light due to organic lesions on the blind person's body, and still a large part of eyeballs can control rotation according to the subjective intention and the target direction; the second determining unit is used for determining the eye-viewing direction of the eyeballs of the blind person by adopting an eyeball tracking technology; and a third determining unit configured to determine a target traveling direction from the first image according to the visual direction, wherein a range of the traveling direction is larger than a range of the target traveling direction.

In order to further accurately determine the visual direction of the blind person so as to provide a more friendly navigation experience according to the visual direction, the second determining unit of the present application includes an acquiring module and a determining module, wherein the acquiring module is used for acquiring the center position of the pupil of the eyeball in the second image; the determining module is used for determining the direction of the center position of the pupil as the visual direction.

In the case that the blind person wants to grasp the target object, the blind person can be prompted according to the combination of tactile feedback and sound feedback, and in some embodiments, the pair of glasses for the blind person further includes a sound pickup device and a voice broadcasting device, where the sound pickup device is used for collecting voice information of the blind person, the voice information characterizes the grasping target object, and the voice broadcasting device is in communication connection with the controller, where the voice broadcasting device is used for performing voice feedback on the blind person according to the obstacle avoidance instruction sent by the controller, and the apparatus further includes a third acquiring unit, a fourth determining unit, a second control unit and a third control unit, where the third acquiring unit is used for acquiring the voice information after acquiring the first image; the fourth determining unit is used for determining a route of the blind person grabbing the target object according to the voice information; the second control unit is used for controlling the voice broadcasting equipment to output second prompt information, wherein the second prompt information is voice prompt information, and the second prompt information is used for prompting the direction of the target object of the blind person; the third control unit is used for controlling the touch feedback device to output third prompt information, wherein the third prompt information is information of touch prompts, and the third prompt information is used for prompting the blind person of the direction of the target object.

The device also comprises a fifth determining unit, a fourth control unit and a fifth control unit, wherein the fifth determining unit is used for determining whether the gradient change exists in the travelling route of the blind person in the travelling direction according to the first image after the first image is acquired; the fourth control unit is used for controlling the voice broadcasting equipment to output fourth prompt information under the condition that the traveling route of the blind person in the traveling direction is determined to have gradient change, wherein the fourth prompt information is voice prompt information, and the fourth prompt information is used for prompting the blind person to have gradient change in the traveling route in the traveling direction; and the fifth control unit is used for controlling the touch feedback device to output fifth prompting information under the condition that the traveling route of the blind person in the traveling direction is determined to have gradient change, wherein the fifth prompting information is information of touch prompting, and the fifth prompting information is used for prompting the blind person to have gradient change in the traveling route in the traveling direction.

The control device of the blind glasses comprises a processor and a memory, wherein the first acquisition unit, the first determination unit, the first control unit and the like are all stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions. The modules are all located in the same processor; alternatively, the above modules may be located in different processors in any combination.

The processor includes a kernel, and the kernel fetches the corresponding program unit from the memory. The inner core can be provided with one or more than one inner core parameters, and the problem that the blind person navigation glasses in the prior art have poor reminding effect on the blind person in a voice navigation mode is solved by adjusting the inner core parameters.

The memory may include volatile memory, random Access Memory (RAM), and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM), among other forms in computer readable media, the memory including at least one memory chip.

The embodiment of the invention provides a computer readable storage medium, which comprises a stored program, wherein the program is used for controlling equipment where the computer readable storage medium is positioned to execute the control method of the blind glasses.

Specifically, the control method of the glasses for the blind comprises the following steps:

The embodiment of the invention provides a processor, which is used for running a program, wherein the control method of the blind glasses is executed when the program runs.

The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored in the memory and can run on the processor, wherein the processor realizes the control method steps of the glasses for the blind when executing the program. The device herein may be a server, PC, PAD, cell phone, etc.

The present application also provides a computer program product adapted to perform a program of steps of a control method for initializing at least blindman glasses when the program is executed on a data processing device.

It will be appreciated by those skilled in the art that the modules or steps of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than that shown or described herein, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.

It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.

From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

1) The blind glasses of this application have added touch feedback device in blind glasses, when there is the barrier, the equipment of broadcasting has not need to have been reported in waiting for the voice, can in time indicate that there is the barrier in the blind place ahead through the touch feedback, avoid appearing the voice command and have not been reported complete barrier and just touched the condition, in addition, the touch of blind person is more sensitive in general, consequently, if the blind person is in more noisy environment, also need not use the voice to indicate the blind person, can let the blind person in time perception suggestion through the touch feedback, can let the blind person feel clear suggestion, and then improved the warning effect of blind person's glasses.

2) According to the blind glasses control method, the touch feedback device is added into the blind glasses, when obstacles exist, the situation that the obstacles exist in front of the blind can be timely prompted through the touch feedback without being broadcasted by the equal voice broadcasting equipment, and the situation that the complete obstacles are touched when voice instructions are not broadcasted is avoided.

3) According to the control device for the glasses for the blind, the touch feedback device is added in the glasses for the blind, when obstacles are present, the situation that the obstacles are present in front of the blind and the complete obstacles are touched when the voice broadcasting equipment is used for broadcasting the obstacles is not needed, the situation that the voice instructions are not broadcast yet is avoided, in addition, the touch of the blind is sensitive, if the blind is in a noisy environment, the blind is not needed to be prompted by voice, the blind can timely perceive the prompt through the touch feedback, the blind can feel clear prompt, and the prompting effect of the glasses for the blind is improved.

The foregoing description is only of the preferred embodiments of the present application and is not intended to limit the same, but rather, various modifications and variations may be made by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A pair of spectacles for the blind, comprising:

the first image acquisition device is used for acquiring a first image of the blind person in the travelling direction;

the controller is in communication connection with the first image acquisition equipment and is used for determining whether an obstacle exists in the travelling direction of the blind person according to the first image;

the touch feedback device is in communication connection with the controller, and is used for performing touch feedback on the blind person according to an obstacle avoidance instruction sent by the controller, wherein the obstacle avoidance instruction is an instruction sent by the controller when the controller determines that the obstacle exists in the travelling direction of the blind person.

2. The pair of blindman glasses according to claim 1, wherein the blindman glasses further comprise:

the second image acquisition device is in communication connection with the controller and is used for acquiring a second image of the eyeballs of the blind person, and the controller determines the visual direction of the blind person according to the second image.

3. The pair of blindman glasses according to claim 1, wherein the blindman glasses further comprise:

the sound pickup device is used for collecting voice information of the blind person, wherein the voice information characterizes a grabbing target object;

and the voice broadcasting equipment is in communication connection with the controller, and is used for carrying out voice feedback on the blind person according to the obstacle avoidance instruction sent by the controller.

4. The pair of blind eyeglasses according to claim 1, wherein said tactile feedback means comprises one or more of pressure feedback means, temperature feedback means and vibration feedback means.

5. A method of controlling the glasses for blind persons according to any one of claims 1 to 4, wherein the method comprises:

acquiring a first image, wherein the first image is an image of the blind person in the travelling direction;

determining whether the obstacle exists in the first image by adopting a target detection algorithm, wherein the target detection algorithm comprises one or more of an R-CNN algorithm, a YOLO algorithm and an SSD algorithm;

and under the condition that the obstacle exists in the first image, controlling the touch feedback device to output first prompt information, wherein the first prompt information is information of touch prompt and is used for prompting the blind person that the obstacle exists in the travelling direction.

6. The method of claim 5, wherein the blindman glasses further comprise a second image acquisition device in communication with the controller, the second image acquisition device for acquiring a second image of the blindman's eye, wherein the controller determines the blindman's visual direction from the second image, the method further comprising, after acquiring the first image:

acquiring the second image, wherein the second image is an image of the eyeball of the blind person;

adopting an eyeball tracking technology to determine the visual direction of the eyeballs of the blind person;

and determining a target traveling direction from the first image according to the visual direction, wherein the range of the traveling direction is larger than that of the target traveling direction.

7. The method of claim 6, wherein determining the eye-ward direction of the eye of the blind using eye tracking techniques comprises:

acquiring the center position of the pupil of the eyeball in the second image;

and determining the direction in which the central position of the pupil is located as the visual direction.

8. The method of claim 5, wherein the pair of blindman glasses further comprises a sound pickup device and a voice broadcasting device, wherein the sound pickup device is configured to collect voice information of the blindman, wherein the voice information characterizes a grabbing target object, and wherein the voice broadcasting device is communicatively connected to the controller, wherein the voice broadcasting device is configured to perform voice feedback on the blindman according to the obstacle avoidance instruction sent by the controller, and after the first image is acquired, the method further comprises:

Acquiring the voice information;

determining a route for the blind person to grasp the target object according to the voice information;

controlling the voice broadcasting equipment to output second prompt information, wherein the second prompt information is information of voice prompt, and the second prompt information is used for prompting the blind person of the direction of the target object;

and controlling the touch feedback device to output third prompt information, wherein the third prompt information is information of touch prompts, and the third prompt information is used for prompting the blind person of the direction of the target object.

9. The method of claim 8, wherein after acquiring the first image, the method further comprises:

determining whether a travel route in the travel direction of the blind person has gradient change according to the first image;

controlling the voice broadcasting equipment to output fourth prompt information under the condition that the traveling route of the blind person in the traveling direction is determined to have gradient change, wherein the fourth prompt information is voice prompt information and is used for prompting the blind person to have gradient change in the traveling route in the traveling direction;

and under the condition that the traveling route of the blind person in the traveling direction is determined to have gradient change, controlling the haptic feedback device to output fifth prompting information, wherein the fifth prompting information is information of haptic prompting, and the fifth prompting information is used for prompting the blind person to have gradient change in the traveling route in the traveling direction.

10. A control device for glasses for blind persons according to any one of claims 1 to 4, characterized in that the device comprises:

a first acquisition unit configured to acquire a first image, wherein the first image is an image in a traveling direction of the blind person;

a first determining unit configured to determine whether the obstacle exists in the first image by using an object detection algorithm, where the object detection algorithm includes one or more of an R-CNN algorithm, a YOLO algorithm, and an SSD algorithm;

the first control unit is used for controlling the touch feedback device to output first prompt information under the condition that the obstacle exists in the first image, wherein the first prompt information is information of touch prompt, and the first prompt information is used for prompting the blind person that the obstacle exists in the travelling direction.