CN109602585B - Blind guiding glasses and anti-collision early warning method thereof - Google Patents

Blind guiding glasses and anti-collision early warning method thereof Download PDF

Info

Publication number
CN109602585B
CN109602585B CN201811451997.7A CN201811451997A CN109602585B CN 109602585 B CN109602585 B CN 109602585B CN 201811451997 A CN201811451997 A CN 201811451997A CN 109602585 B CN109602585 B CN 109602585B
Authority
CN
China
Prior art keywords
blind
image
obstacle
obstacles
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811451997.7A
Other languages
Chinese (zh)
Other versions
CN109602585A (en
Inventor
金守峰
高磊
陈蓉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Polytechnic University
Original Assignee
Xian Polytechnic University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Polytechnic University filed Critical Xian Polytechnic University
Priority to CN201811451997.7A priority Critical patent/CN109602585B/en
Publication of CN109602585A publication Critical patent/CN109602585A/en
Application granted granted Critical
Publication of CN109602585B publication Critical patent/CN109602585B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H3/06—Walking aids for blind persons
    • A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/24—Reminder alarms, e.g. anti-loss alarms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
    • A61H2201/01—Constructive details
    • A61H2201/0173—Means for preventing injuries
    • A61H2201/0184—Means for preventing injuries by raising an alarm
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2201/00—Characteristics of apparatus not provided for in the preceding codes
    • A61H2201/16—Physical interface with patient
    • A61H2201/1602—Physical interface with patient kind of interface, e.g. head rest, knee support or lumbar support
    • A61H2201/165—Wearable interfaces

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Pain & Pain Management (AREA)
  • Epidemiology (AREA)
  • Rehabilitation Therapy (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Image Analysis (AREA)
  • Rehabilitation Tools (AREA)

Abstract

本发明公开的一种导盲眼镜，包括眼镜架及安装在眼镜架上的镜片，眼镜架前侧分别安装有摄像头、照明灯、光线传感器，眼镜架一条支腿端部安装有耳机，眼镜架的另一条支腿端部外侧安装有控制按键，还包括控制盒。本发明公开的一种导盲眼镜的防撞预警方法，具体按照以下步骤实施；步骤1、拍摄多张连续图像；步骤2、对多张连续图像进行预处理；步骤3、识别盲道，若无法识别则重新拍摄照片返回步骤1重新开始；步骤4、对盲道区域内的障碍物进行识别，若无法识别则重新拍摄照片返回步骤1重新开始；步骤5、计算障碍物与导盲眼镜使用者之间的距离，并对导盲眼镜使用者预警提醒。本发明的一种导盲眼镜，能够随身携带、便于操控、通过防撞预警方法能够有效地提醒使用者与障碍物之间的距离。

Figure 201811451997

The invention discloses a kind of guide glasses for the blind, comprising a spectacle frame and a lens mounted on the spectacle frame. A camera, a lighting lamp and a light sensor are respectively installed on the front side of the spectacle frame. An earphone is installed at the end of one leg of the spectacle frame. A control button is installed on the outside of the end of the other outrigger, and a control box is also included. A collision avoidance warning method for guide glasses disclosed in the present invention is specifically implemented according to the following steps: step 1, shooting a plurality of consecutive images; step 2, preprocessing the plurality of consecutive images; step 3, identifying blind lanes, if unable If it is recognized, take a photo again and return to step 1 to start again; step 4, identify the obstacles in the blind lane area, if it cannot be recognized, take a new photo and return to step 1 to start again; step 5, calculate the relationship between the obstacle and the guide glasses user The distance between them, and warnings to guide glasses users. The blind guide glasses of the present invention can be carried around, easy to operate, and can effectively remind the user of the distance between the obstacle and the obstacle through the anti-collision warning method.

Figure 201811451997

Description

Blind guiding glasses and anti-collision early warning method thereof
Technical Field
The invention belongs to the technical field of wearable equipment, and particularly relates to blind guiding glasses and an anti-collision early warning method thereof.
Background
The visual system is one of the most important systems for human perception of the objective world, and the human obtains more than 80% of the external information. Therefore, the lack or decline of visual information makes it difficult for visually impaired people to perceive the surrounding environment information, and the free travel is hindered, and the aspects of life self-care and the like are also limited, which brings much inconvenience and trouble to the daily life of the group and is a main factor causing the reduction of the quality of life. The world health organization predicts that the number of global vision-impaired people will increase to 7500 thousands of people in 2020, the sixth census of China and the second disabled people survey data statistics of China show that the number of the vision-impaired people in China reaches 877 thousands of people, and the world is the most country. And as the aging population of China continuously increases, the number of the visually impaired people also increases sharply, so that the problem of safe trip of the visually impaired people is solved, the life quality of the visually impaired people is improved, and the visually impaired people are attracted by social attention.
At present, the travel of the visually impaired people mainly depends on the blind road and the handheld blind stick, but the limitation of the blind road area and the space information amount provided by the blind stick can not enable the visually impaired people to safely travel. In addition, only a few visually-impaired people use the guide dog to lead a trip, but the guide dog training learning time is long, the cost is high, and the actual requirements of visually-impaired people are difficult to meet. The blind guiding robot integrates various sensors on the movable carrier, so that the blind guiding robot can sense environmental information and guide visually impaired people to safely go out. However, the blind guiding robot has low capability of adapting to the environment and limited application range. The blind guiding and walking aid device in the prior art is often provided with a movable carrier, or cannot be carried about due to large volume, or brings unnecessary trouble to the use of the visually impaired people due to troublesome operation and the like.
Disclosure of Invention
The invention aims to provide blind guiding glasses which can be carried about, are convenient to operate and control and effectively remind a user of the distance between the user and an obstacle.
The technical scheme adopted by the invention is as follows: a pair of blind guiding glasses comprises a glasses frame and lenses arranged on the glasses frame, wherein the front side of the glasses frame is respectively provided with a camera, a lighting lamp and a light sensor, the camera is positioned at the center of the front side of the glasses frame, the end part of one supporting leg of the glasses frame is provided with an earphone, the outer side of the end part of the other supporting leg of the glasses frame is provided with a control key,
also comprises a control box, a main controller, an image processor, a battery, a vibration module, a voice module, a Bluetooth communication module and a WiFi communication module which are connected with the main controller are arranged in the control box,
the camera is connected with an image processor in the control box through a video line, the control keys and the earphone are respectively connected with the main controller and the voice module in the control box through data lines, the illuminating lamp and the light sensor are both connected with the main controller in the control box through the data lines, and the control box controls the opening and closing of the illuminating lamp according to light intensity signals transmitted by the light sensor.
The present invention is also characterized in that,
different key areas are divided on the control keys, and braille characters are arranged on each key area, so that the operation of the blind is facilitated.
The spectacle frame is made of hard aluminum alloy materials, so that the spectacle frame is convenient to be attached to the face without falling off.
The second technical scheme adopted by the invention is that the anti-collision early warning method for the blind guiding glasses is implemented according to the following steps:
step 1, shooting to obtain a plurality of continuous images of the traveling direction of a user of the blind guiding glasses;
step 2, preprocessing a plurality of continuous images, increasing the image contrast and reducing the image noise;
step 3, extracting blind road characteristics according to the color characteristics and the texture characteristics of the preprocessed image, identifying the blind road, re-shooting the picture if the blind road characteristics cannot be extracted, returning to the step 1, and otherwise, performing the step 4;
step 4, identifying obstacles in the blind road area, determining static obstacles and dynamic obstacles, if the static obstacles and the dynamic obstacles cannot be determined, re-shooting the picture, returning to the step 1 to start, and otherwise, performing the step 5;
and 5, calculating the distance between the obstacle and the blind guiding glasses user according to the static obstacle and the dynamic obstacle determined in the step 4, and giving an early warning to the blind guiding glasses user.
The preprocessing process in the step 2 is to process a plurality of continuous images by adopting a histogram equalization and homomorphic filtering algorithm.
Step 3 is specifically implemented according to the following steps:
step 3.1, converting the RGB model of the preprocessed image into a Lab color space model by adopting a color space conversion model to obtain b-component characteristics of the blind road image in the Lab color space model, wherein the color space conversion model is as follows:
Figure GDA0003366905760000031
wherein X, Y, Z are the three stimulus values of the image, X0、Y0、Z0Three stimulus values, L, for CTE standard illumination*Representing the brightness of the image, a*、b*Representing image chromaticity;
step 3.2, adopting a gray level co-occurrence matrix, and selecting four direction angles of 0 degree, 45 degrees, 90 degrees and 135 degrees to perform statistics on the image to obtain four gray level co-occurrence matrices of P0 degrees, P45 degrees, P90 degrees and P135 degrees respectively;
step 3.3, combining the b-component characteristic of the blind road image obtained in the step 3.1 and the four gray level co-occurrence matrixes respectively of P0 degrees, P45 degrees, P90 degrees and P135 degrees obtained in the step 3.2, and segmenting the image by adopting a maximum inter-class variance method to preliminarily obtain a blind road region;
and 3.4, extracting blind channel edge characteristics from the blind channel region obtained primarily in the step 3.3 by adopting a Canny operator, and extracting the blind channel edge characteristics through Hough transformation to obtain a final complete blind channel region.
Step 4 is specifically implemented according to the following steps:
step 4.1, determining static obstacles
Step 4.1.1, neglecting the upper area of the blind road area in each image, and extracting the blind road area;
step 4.1.2, calculating the gray values of the blind road region obtained in the step 4.1.1 in the vertical direction and the horizontal direction, calculating the entropy of the blind road region obtained in the step 4.1.1 to judge whether a static obstacle exists in the blind road region, and judging whether the static obstacle exists when the gray values of the blind road region in the vertical direction and the horizontal direction are both smaller than the entropy of the blind road region;
step 4.2, determining dynamic obstacle
Step 4.2.1, detecting every three adjacent images by adopting a three-frame difference method, wherein the formula of the three-frame difference method is as follows
Figure GDA0003366905760000041
Wherein, Fk-1(x，y)、Fk(x，y)、Fk+1(x, y) is the k-1, k +1 frame image in succession, G1(x,y)、G2(x, y) is the frame-differenced image;
step 4.2.2, image G after frame difference1(x,y)、G2(x, y) selecting a threshold value T for binarization processing, wherein the binarization processing process comprises the following steps:
Figure GDA0003366905760000042
Figure GDA0003366905760000043
wherein T is a threshold value and D1(x，y)、D2(x, y) is the binarized image,
wherein the threshold T is based on the frame-corrupted image G1(x,y)、G2(x, y) selecting the distribution of gray values;
step 4.2.3, the binarized image D obtained in the step 4.2.21(x，y)、D2(x, y) performing a logical AND operation:
Figure GDA0003366905760000051
in the formula, R (x, y) is an image obtained after AND operation;
and 4.2.4, performing gray projection on the image obtained after the AND operation in the horizontal and vertical directions by adopting a gray projection algorithm to obtain two curves of the image obtained after the AND operation in the two directions, performing cross-correlation calculation on the two curves by using a cross-correlation function to obtain cross-correlation curves of the two curves, and determining the dynamic obstacle by using a peak point on the cross-correlation curves.
Step 5 is specifically implemented according to the following steps:
step 5.1, substituting the static obstacle and dynamic obstacle information obtained in the step 4 into a distance measurement formula to calculate the distance between the static obstacle and the dynamic obstacle and a user of the blind guiding glasses, wherein the distance measurement formula is as follows:
Figure GDA0003366905760000052
h is the installation height of the camera, H is the height of a static obstacle or a dynamic obstacle, f is the focal length of the camera, u, v, u0，v0，ax，ayThe internal parameters of the camera are taken as the parameters;
step 5.2, according to the distance between the static barrier and the dynamic barrier calculated in the step 5.1 and the blind guiding glasses user, giving an early warning to the blind guiding glasses user: when the distance is less than 3 meters, the blind guiding glasses user is in an unsafe area, the control box sends out a warning about the existence of the obstacle to the blind guiding glasses user through the voice module and the vibration module, when the distance is greater than 7 meters, the control box sends out a safe voice signal, and when the distance is greater than 3 meters and less than 7 meters, the system sends out a warning signal about the existence of the obstacle.
The invention has the beneficial effects that: the blind guiding glasses are simple in preparation process and low in cost, are suitable for large-scale production and popularization, can be carried about, are convenient to operate and control, can effectively remind a user of the distance between the user and an obstacle through an anti-collision early warning method, avoid the user from colliding with the obstacle on a blind road, and bring great convenience to the user in traveling.
Drawings
FIG. 1 is a view of the structure of the appearance of a pair of blind-guiding spectacles according to the present invention;
fig. 2 is a flow chart of an anti-collision early warning method of blind guide glasses according to the present invention.
The device comprises an earphone 1, an illuminating lamp 2, a camera 3, a lens 4, an eyeglass frame 5, a control box 6, a control button 7 and a light sensor 8.
Detailed Description
The following detailed description of the embodiments of the invention refers to the accompanying drawings and accompanying claims.
The invention provides a pair of blind guiding glasses, as shown in figure 1, comprising a glasses frame 5 and a lens 4 arranged on the glasses frame 5, wherein the front side of the glasses frame 5 is respectively provided with a camera 3, a lighting lamp 2 and a light sensor 8, the camera 3 is positioned at the center of the front side of the glasses frame 5, the end part of one supporting leg of the glasses frame 5 is provided with an earphone 1, the outer side of the end part of the other supporting leg of the glasses frame 5 is provided with a control key 7,
also comprises a control box 6, a main controller, an image processor, a battery, a vibration module, a voice module, a Bluetooth communication module and a WiFi communication module which are connected with the main controller are arranged in the control box 6,
camera 3 passes through the video line and is connected with the image processor in the control box 6, control button 7, earphone 1 passes through the master controller in data line and the control box 6 respectively, voice module is connected, light 2, light sensor 8 all is connected with the master controller in the control box 6 through the data line, control box 6 is closed according to opening of the strong and weak signal control light 2 of light sensor 8 transmission, thereby when light is dark, control light 2 is opened supplementary camera 3 and is shot clear photo.
Furthermore, different key areas are divided on the control keys 7, and braille characters are arranged on each key area, so that the operation of the blind is facilitated.
Furthermore, the spectacle frame 5 is made of hard aluminum alloy materials, so that the spectacle frame 5 is convenient to be attached to the face without falling off.
An anti-collision early warning method based on the blind-guiding glasses is specifically implemented according to the following steps as shown in fig. 2:
step 1, shooting to obtain a plurality of continuous images of the traveling direction of a user of the blind guiding glasses;
step 2, preprocessing a plurality of continuous images, increasing the image contrast and reducing the image noise;
step 3, extracting blind road characteristics according to the color characteristics and the texture characteristics of the preprocessed image, identifying the blind road, re-shooting the picture if the blind road characteristics cannot be extracted, returning to the step 1, and otherwise, performing the step 4;
step 4, identifying obstacles in the blind road area, determining static obstacles and dynamic obstacles, if the static obstacles and the dynamic obstacles cannot be determined, re-shooting the picture, returning to the step 1 to start, and otherwise, performing the step 5;
and 5, calculating the distance between the obstacle and the blind guiding glasses user according to the static obstacle and the dynamic obstacle determined in the step 4, and giving an early warning to the blind guiding glasses user.
Further, the preprocessing process in step 2 is to process a plurality of continuous images by using histogram equalization and homomorphic filtering algorithm.
Step 3 is specifically implemented according to the following steps:
step 3.1, converting the RGB model of the preprocessed image into a Lab color space model by adopting a color space conversion model to obtain b-component characteristics of the blind road image in the Lab color space model, wherein the color space conversion model is as follows:
Figure GDA0003366905760000081
wherein X, Y, Z are the three stimulus values of the image, X0、Y0、Z0Three stimulus values, L, for CTE standard illumination*Representing the brightness of the image, a*、b*Representing image chromaticity;
step 3.2, adopting a gray level co-occurrence matrix, and selecting four direction angles of 0 degree, 45 degrees, 90 degrees and 135 degrees to perform statistics on the image to obtain four gray level co-occurrence matrices of P0 degrees, P45 degrees, P90 degrees and P135 degrees respectively;
step 3.3, combining the b-component characteristic of the blind road image obtained in the step 3.1 and the four gray level co-occurrence matrixes respectively of P0 degrees, P45 degrees, P90 degrees and P135 degrees obtained in the step 3.2, and segmenting the image by adopting a maximum inter-class variance method to preliminarily obtain a blind road region;
and 3.4, extracting blind channel edge characteristics from the blind channel region obtained primarily in the step 3.3 by adopting a Canny operator, and extracting the blind channel edge characteristics through Hough transformation to obtain a final complete blind channel region.
Step 4 is specifically implemented according to the following steps:
step 4.1, determining static obstacles
Step 4.1.1, neglecting the upper area of the blind road area in each image, and extracting the blind road area;
step 4.1.2, calculating the gray values of the blind road region obtained in the step 4.1.1 in the vertical direction and the horizontal direction, calculating the entropy of the blind road region obtained in the step 4.1.1 to judge whether a static obstacle exists in the blind road region, and judging whether the static obstacle exists when the gray values of the blind road region in the vertical direction and the horizontal direction are both smaller than the entropy of the blind road region;
step 4.2, determining dynamic obstacle
Step 4.2.1, detecting every three adjacent images by adopting a three-frame difference method, wherein the formula of the three-frame difference method is as follows
G1(x，y)＝|Fk(x，y)-Fk-1(x，y)| (2)
G2(x，y)＝|Fk+1(x，y)-Fk(x，y)|
Wherein, Fk-1(x，y)、Fk(x，y)、Fk+1(x, y) is the k-1, k +1 frame image in succession, G1(x,y)、G2(x, y) is the frame-differenced image;
step 4.2.2, image G after frame difference1(x,y)、G2(x, y) selecting a threshold value T for binarization processing, wherein the binarization processing process comprises the following steps:
Figure GDA0003366905760000091
Figure GDA0003366905760000092
in the formula (I), the compound is shown in the specification,t is a threshold value, D1(x，y)、D2(x, y) is the binarized image,
wherein the threshold T is based on the frame-corrupted image G1(x,y)、G2(x, y) selecting the distribution of gray values;
step 4.2.3, the binarized image D obtained in the step 4.2.21(x，y)、D2(x, y) performing a logical AND operation:
Figure GDA0003366905760000093
in the formula, R (x, y) is an image obtained after AND operation;
and 4.2.4, performing gray projection on the image obtained after the AND operation in the horizontal and vertical directions by adopting a gray projection algorithm to obtain two curves of the image obtained after the AND operation in the two directions, performing cross-correlation calculation on the two curves by using a cross-correlation function to obtain cross-correlation curves of the two curves, and determining the dynamic obstacle by using a peak point on the cross-correlation curves.
Step 5 is specifically implemented according to the following steps:
step 5.1, substituting the static obstacle and dynamic obstacle information obtained in the step 4 into a distance measurement formula to calculate the distance between the static obstacle and the dynamic obstacle and a user of the blind guiding glasses, wherein the distance measurement formula is as follows:
Figure GDA0003366905760000094
h is the installation height of the camera, H is the height of a static obstacle or a dynamic obstacle, f is the focal length of the camera, u, v, u0，v0，ax，ayThe internal parameters of the camera are taken as the parameters;
step 5.2, according to the distance between the static barrier and the dynamic barrier calculated in the step 5.1 and the blind guiding glasses user, giving an early warning to the blind guiding glasses user: when the distance is less than 3 meters, the blind guiding glasses user is in an unsafe area, the control box 6 sends out a warning about the existence of the obstacle to the blind guiding glasses user through the voice module and the vibration module, when the distance is greater than 7 meters, the control box 6 sends out a safe voice signal, and when the distance is greater than 3 meters and less than 7 meters, the system sends out a warning signal about the existence of the obstacle.
The blind guiding glasses are simple in preparation process and low in cost, are suitable for large-scale production and popularization, can be carried about, are convenient to operate and control, can effectively remind a user of the distance between the user and an obstacle through an anti-collision early warning method, avoid the user from colliding with the obstacle on a blind road, and bring great convenience to the user in traveling.
The invention discloses an anti-collision early warning method for blind guiding glasses, which is a blind road identification detection algorithm integrating color features and texture features. Extracting yellow b component of the blind road in Lab color space, segmenting the blind road image by a maximum inter-class variance method, preliminarily extracting edge points of the blind road by adopting a Canny operator through texture features of the blind road, further optimizing the edge points of the blind road through Hough transformation, and simultaneously fusing segmented blind road regions to effectively identify the blind road.
Because the environment along the blind road is complex, and the appearance of the obstacle has unpredictability and uncertainty, the method adopts the single-frame-based image to process the obstacle right ahead, establishes an interested area for the acquired image, namely ignores the area above the blind road area in each image, extracts the blind road area to reduce the search range of the obstacle, further reduces the area of the obstacle by respectively carrying out edge detection in two directions on the area, and finally calculates the entropy of the image to judge whether the area really has the obstacle.
The invention adopts sequence-based images to process the moving obstacles suddenly appearing on both sides of the blind road, and combines a gray projection algorithm to detect the moving obstacles on the basis of a three-frame difference method.
On the basis of realizing the detection and positioning of the cataract obstacle in the blind road area, the invention designs early warning reminding, determines the geometric relation between a world coordinate system and an image coordinate system through the calibration of a vision system on the premise of assuming that the road surface of the blind road is horizontal, estimates the actual distance between the visually impaired and the obstacle through distance measurement, and sends out a safe voice signal, an early warning signal and a warning signal in 3 different forms according to the actual distance, thereby effectively ensuring the personal safety of visually impaired people.

Claims (1)

1.一种导盲眼镜的防撞预警方法，其特征在于，采用一种导盲眼镜，包括眼镜架(5)及安装在眼镜架(5)上的镜片(4)，所述眼镜架(5)前侧分别安装有摄像头(3)、照明灯(2)、光线传感器(8)，所述摄像头(3)位于眼镜架(5)前侧中心处，所述眼镜架(5)一条支腿端部安装有耳机(1)，所述眼镜架(5)的另一条支腿端部外侧安装有控制按键(7)，1. a kind of anti-collision warning method of guide glasses, is characterized in that, adopts a kind of guide glasses, comprises spectacle frame (5) and the lens (4) installed on spectacle frame (5), described spectacle frame ( 5) A camera (3), a lighting lamp (2), and a light sensor (8) are respectively installed on the front side, the camera (3) is located at the center of the front side of the spectacle frame (5), and the spectacle frame (5) has one support. An earphone (1) is installed at the end of the leg, and a control button (7) is installed outside the end of the other leg of the spectacle frame (5), 还包括控制盒(6)，所述控制盒(6)内安装有主控器以及与主控器相连的图像处理器、电池、振动模块、语音模块、蓝牙通信模块及WiFi通信模块，Also includes a control box (6), in which a main controller and an image processor, a battery, a vibration module, a voice module, a Bluetooth communication module and a WiFi communication module connected to the main controller are installed, 所述摄像头(3)通过视频线与控制盒(6)内的图像处理器连接，所述控制按键(7)、耳机(1)分别通过数据线与控制盒(6)内的主控器、语音模块连接，所述照明灯(2)、光线传感器(8)均通过数据线与控制盒(6)内的主控器连接，所述控制盒(6)根据光线传感器(8)传递的光线强弱信号控制照明灯(2)的开启关闭；The camera head (3) is connected to the image processor in the control box (6) through a video cable, and the control button (7) and the earphone (1) are respectively connected to the main controller in the control box (6) through a data cable, The voice module is connected, and the lighting lamp (2) and the light sensor (8) are both connected to the main controller in the control box (6) through a data cable, and the control box (6) is based on the light transmitted by the light sensor (8). The strong and weak signals control the opening and closing of the lighting lamp (2); 所述控制按键(7)上划分有不同按键区，每个按键区上均设有盲文文字，方便盲人操作；The control keys (7) are divided into different key areas, and each key area is provided with Braille characters, which is convenient for the blind to operate; 所述眼镜架(5)采用硬质铝合金材料，便于眼镜架(5)与脸部贴合不掉落；The spectacle frame (5) is made of hard aluminum alloy material, so that the spectacle frame (5) can fit with the face and not fall off; 具体按照以下步骤实施：Specifically, follow the steps below: 步骤1、拍摄获得导盲眼镜使用者行进方向的多张连续图像；Step 1. Shoot and obtain a plurality of consecutive images of the direction of travel of the guide glasses user; 步骤2、对多张连续图像进行预处理，增加图像对比度，降低图像噪声；Step 2. Preprocess multiple consecutive images to increase image contrast and reduce image noise; 其中，预处理过程为采用直方图均衡化、同态滤波算法对多张连续图像进行处理；Among them, the preprocessing process is to use histogram equalization and homomorphic filtering algorithm to process multiple consecutive images; 步骤3、根据预处理后的图像的颜色特征、纹理特征提取盲道特征，识别盲道，若无法提取到盲道特征则重新拍摄照片返回步骤1开始，否则进行步骤4；Step 3. Extract the blind lane feature according to the color feature and texture feature of the preprocessed image, identify the blind lane, and if the blind lane feature cannot be extracted, retake the photo and return to step 1 to start, otherwise go to step 4; 步骤3.1、采用颜色空间转换模型将预处理后的图像的RGB模型转换为Lab颜色空间模型，得到Lab颜色空间模型中盲道图像的b分量特征，所述颜色空间转换模型为：Step 3.1, adopt the color space conversion model to convert the RGB model of the preprocessed image into the Lab color space model, and obtain the b-component feature of the blind image in the Lab color space model, and the color space conversion model is:
Figure FDA0003366905750000021
Figure FDA0003366905750000021
 其中，X、Y、Z是图像的三个刺激值，X0、Y0、Z0是CTE标准照明的三个刺激值，L* 表示图像明度，a* 、 b* 表示图像色度；Among them, X, Y, Z are the three stimulus values of the image, X 0 , Y 0 , Z 0 are the three stimulus values of the CTE standard illumination, L * represents the image brightness, a * , b * represents the image chromaticity; 步骤3.2、采用灰度共生矩阵，选择0°、45°、90°、135°四个方向角度对图像进行统计，得到四个分别为P0°、P45°、P90°、P135°的灰度共生矩阵；Step 3.2. Using the grayscale co-occurrence matrix, select four directions of 0°, 45°, 90°, and 135° to count the image, and obtain four grayscale co-occurrences of P0°, P45°, P90°, and P135° respectively. matrix; 步骤3.3、结合经步骤3.1得到的盲道图像的b分量特征和经步骤3.2得到的四个分别为P0°、P45°、P90°、P135°的灰度共生矩阵，采用最大类间方差法对图像进行分割，初步得到盲道区域；Step 3.3: Combine the b-component feature of the blind channel image obtained in step 3.1 and the four gray-level co-occurrence matrices of P0°, P45°, P90°, and P135° obtained by step 3.2, and use the maximum inter-class variance method to analyze the image. Segmentation is performed to initially obtain the blind road area; 步骤3.4、对经步骤3.3初步得到的盲道区域采用Canny算子提取盲道边缘特征，再通过Hough变换提取盲道边缘特征，得到最终完整的盲道区域；Step 3.4, using the Canny operator to extract the edge features of the blind track for the blind track area initially obtained in step 3.3, and then extracting the edge features of the blind track through Hough transform to obtain the final complete blind track area; 步骤4、对盲道区域内的障碍物进行识别，确定静态障碍物、动态障碍物，若无法确定到静态障碍物、动态障碍物，则重新拍摄照片返回步骤1开始，否则进行步骤5；Step 4. Identify the obstacles in the blind lane area, and determine the static obstacles and dynamic obstacles. If the static obstacles and dynamic obstacles cannot be determined, take a photo again and return to step 1 to start, otherwise go to step 5; 步骤4.1、确定静态障碍物Step 4.1. Determine static obstacles 步骤4.1.1、忽略每一张图像中盲道区域的上方区域，提取盲道区域；Step 4.1.1. Ignore the area above the blind area in each image, and extract the blind area; 步骤4.1.2、计算经步骤4.1.1得到的盲道区域的垂直和水平两个方向上的灰度值，计算经步骤4.1.1得到的盲道区域的熵来判别盲道区域内是否存在静态障碍物，当盲道区域的垂直和水平两个方向上的灰度值均小于盲道区域的熵，则存在静态障碍物；Step 4.1.2. Calculate the gray values in the vertical and horizontal directions of the blind lane area obtained in step 4.1.1, and calculate the entropy of the blind lane area obtained in step 4.1.1 to determine whether there are static obstacles in the blind lane area. , when the gray values in the vertical and horizontal directions of the blind road area are both smaller than the entropy of the blind road area, there is a static obstacle; 步骤4.2、确定动态障碍物Step 4.2. Determine dynamic obstacles 步骤4.2.1、对每相邻的三张图像采用三帧差法进行检测，所述的三帧差法公式为Step 4.2.1. Use the three-frame difference method to detect each adjacent three images. The three-frame difference method formula is:
Figure FDA0003366905750000031
Figure FDA0003366905750000031
 其中，Fk-1(x，y)、Fk(x，y)、Fk+1(x，y)为连续的第k-1、k、k+1帧图像，G1(x,y)、G2(x,y)为帧差后的图像；Among them, F k-1 (x, y), F k (x, y), F k+1 (x, y) are consecutive k-1, k, k+1 frames of images, G 1 (x, y), G 2 (x, y) are the images after frame difference; 步骤4.2.2、对帧差后的图像G1(x,y)、G2(x,y)选取阈值T进行二值化处理，所述二值化处理过程为:Step 4.2.2, select the threshold value T to carry out binarization processing to the images G 1 (x, y) and G 2 (x, y) after the frame difference, and the binarization processing process is:
Figure FDA0003366905750000032
Figure FDA0003366905750000032
Figure FDA0003366905750000033
Figure FDA0003366905750000033
 式中，T为阈值，D1(x，y)、D2(x，y)为二值化后的图像，In the formula, T is the threshold, D 1 (x, y), D 2 (x, y) are the binarized images, 其中阈值T根据帧差后的图像G1(x,y)、G2(x,y)灰度值的分布选取；The threshold T is selected according to the distribution of the gray value of the images G 1 (x, y) and G 2 (x, y) after the frame difference; 步骤4.2.3、将经步骤4.2.2得到的二值化后的图像D1(x，y)、D2(x，y)进行逻辑“与”运算：Step 4.2.3. Perform a logical AND operation on the binarized images D 1 (x, y) and D 2 (x, y) obtained in Step 4.2.2:
Figure FDA0003366905750000034
Figure FDA0003366905750000034
 式中，R(x，y)为“与”运算后得到的图像；In the formula, R(x, y) is the image obtained after the "AND" operation; 步骤4.2.4、采用灰度投影算法对“与”运算后得到的图像在水平、竖直方向上进行灰度投影，得到“与”运算后得到的图像在两个方向上的两条曲线，再通过互相关函数对两条曲线进行互相关计算，得到这两条曲线的互相关曲线，通过互相关曲线上的峰值点确定动态障碍物；Step 4.2.4. Use the grayscale projection algorithm to perform grayscale projection on the image obtained after the "AND" operation in the horizontal and vertical directions, and obtain two curves in the two directions of the image obtained after the "AND" operation. Then calculate the cross-correlation of the two curves through the cross-correlation function to obtain the cross-correlation curve of the two curves, and determine the dynamic obstacle by the peak point on the cross-correlation curve; 步骤5、根据步骤4确定的静态障碍物、动态障碍物计算障碍物与导盲眼镜使用者之间的距离，并对导盲眼镜使用者预警提醒；Step 5. Calculate the distance between the obstacle and the user of the guide glasses according to the static obstacles and dynamic obstacles determined in step 4, and give an early warning to the user of the guide glasses; 步骤5.1、将步骤4得到的静态障碍物、动态障碍物信息代入测距公式计算静态障碍物、动态障碍物与导盲眼镜使用者之间的距离，所述测距公式为：Step 5.1. Substitute the static obstacle and dynamic obstacle information obtained in step 4 into the ranging formula to calculate the distance between the static obstacle, the dynamic obstacle and the user of the guide glasses. The ranging formula is:
Figure FDA0003366905750000041
Figure FDA0003366905750000041
 其中，H为摄像头安装高度，h为静态障碍物或动态障碍物的高度，f为摄像头焦距，u，v，u0，v0，ax，ay为摄像头的内参数；Among them, H is the installation height of the camera, h is the height of static obstacles or dynamic obstacles, f is the focal length of the camera, u, v, u 0 , v 0 , a x , a y are the internal parameters of the camera; 步骤5.2、根据步骤5.1计算得到的静态障碍物、动态障碍物与导盲眼镜使用者之间的距离，对导盲眼镜使用者预警提醒：当距离小于3米时，则导盲眼镜使用者处在不安全的区域，控制盒(6)通过语音模块、振动模块对导盲眼镜使用者发出障碍物存在警告，当距离大于7米时，控制盒(6)发出安全语音信号，当距离大于3米和小于7米时，系统发出障碍物存在预警信号。Step 5.2. According to the distance between the static obstacle, dynamic obstacle and the guide glasses user calculated in step 5.1, give an early warning to the guide glasses user: when the distance is less than 3 meters, the guide glasses user will In an unsafe area, the control box (6) sends out a warning of the existence of obstacles to the guide glasses users through the voice module and the vibration module. When the distance is greater than 7 meters, the control box (6) sends out a safe voice signal, and when the distance is greater than 3 meters When the distance is less than 7 meters, the system sends out an early warning signal for the existence of obstacles.
CN201811451997.7A 2018-11-30 2018-11-30 Blind guiding glasses and anti-collision early warning method thereof Active CN109602585B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811451997.7A CN109602585B (en) 2018-11-30 2018-11-30 Blind guiding glasses and anti-collision early warning method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811451997.7A CN109602585B (en) 2018-11-30 2018-11-30 Blind guiding glasses and anti-collision early warning method thereof

Publications (2)

Publication Number Publication Date
CN109602585A CN109602585A (en) 2019-04-12
CN109602585B true CN109602585B (en) 2022-02-22

Family

ID=66006495

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811451997.7A Active CN109602585B (en) 2018-11-30 2018-11-30 Blind guiding glasses and anti-collision early warning method thereof

Country Status (1)

Country Link
CN (1) CN109602585B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110623820A (en) * 2019-07-15 2019-12-31 电子科技大学 A wearable intelligent blind guide device
CN113080550A (en) * 2021-05-12 2021-07-09 新疆爱华盈通信息技术有限公司 Headband for recognizing obstacle and method for recognizing obstacle

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101336856A (en) * 2008-08-08 2009-01-07 西安电子科技大学 Information Acquisition and Transmission Method of Auxiliary Vision System
WO2011083963A2 (en) * 2010-01-06 2011-07-14 (주) 부성 리싸이클링 System and method for guiding the walking direction of the visually impaired using rfid blocks
CN102184550A (en) * 2011-05-04 2011-09-14 华中科技大学 Mobile platform ground movement object detection method
CN103637900A (en) * 2013-12-20 2014-03-19 北京航空航天大学 Intelligent blind guiding stick based on image identification
CN203825313U (en) * 2013-12-16 2014-09-10 智博锐视（北京）科技有限公司 Blind navigation glasses
CN104173174A (en) * 2014-08-22 2014-12-03 河海大学常州校区 Portable walking aid blind guide device and method
CN104778723A (en) * 2015-04-09 2015-07-15 海视英科光电（苏州）有限公司 Method for performing motion detection on infrared image with three-frame difference method
CN106074098A (en) * 2016-06-13 2016-11-09 金英子 A kind of navigation system for visually impaired person
CN106709518A (en) * 2016-12-20 2017-05-24 西南大学 Android platform-based blind way recognition system
RO132133A2 (en) * 2016-03-03 2017-09-29 Gafencu Daniela În Calitate De Tutore A Minorei Gafencu Miruna-Alexandra Individual portable device and method of orientation in motion, meant to be used by persons suffering from severe sight deficiencies
CN107456363A (en) * 2017-08-10 2017-12-12 李海强 Multifunctional guide walking stick for blind people
CN108524209A (en) * 2018-03-30 2018-09-14 江西科技师范大学 Blind-guiding method, system, readable storage medium storing program for executing and mobile terminal

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101336856A (en) * 2008-08-08 2009-01-07 西安电子科技大学 Information Acquisition and Transmission Method of Auxiliary Vision System
WO2011083963A2 (en) * 2010-01-06 2011-07-14 (주) 부성 리싸이클링 System and method for guiding the walking direction of the visually impaired using rfid blocks
CN102184550A (en) * 2011-05-04 2011-09-14 华中科技大学 Mobile platform ground movement object detection method
CN203825313U (en) * 2013-12-16 2014-09-10 智博锐视（北京）科技有限公司 Blind navigation glasses
CN103637900A (en) * 2013-12-20 2014-03-19 北京航空航天大学 Intelligent blind guiding stick based on image identification
CN104173174A (en) * 2014-08-22 2014-12-03 河海大学常州校区 Portable walking aid blind guide device and method
CN104778723A (en) * 2015-04-09 2015-07-15 海视英科光电（苏州）有限公司 Method for performing motion detection on infrared image with three-frame difference method
RO132133A2 (en) * 2016-03-03 2017-09-29 Gafencu Daniela În Calitate De Tutore A Minorei Gafencu Miruna-Alexandra Individual portable device and method of orientation in motion, meant to be used by persons suffering from severe sight deficiencies
CN106074098A (en) * 2016-06-13 2016-11-09 金英子 A kind of navigation system for visually impaired person
CN106709518A (en) * 2016-12-20 2017-05-24 西南大学 Android platform-based blind way recognition system
CN107456363A (en) * 2017-08-10 2017-12-12 李海强 Multifunctional guide walking stick for blind people
CN108524209A (en) * 2018-03-30 2018-09-14 江西科技师范大学 Blind-guiding method, system, readable storage medium storing program for executing and mobile terminal

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
导盲机器人的交通标志视觉识别技术研究;汤智超;《计算机技术与发展》;20140930;第24卷(第9期);第23-27页 *

Also Published As

Publication number Publication date
CN109602585A (en) 2019-04-12

Similar Documents

Publication Publication Date Title
Jie et al. A new traffic light detection and recognition algorithm for electronic travel aid
US9704060B2 (en) Method for detecting traffic violation
CN101847206B (en) Pedestrian traffic statistical method and system based on traffic monitoring facilities
US20050275718A1 (en) Apparatus and method for identifying surrounding environment by means of image processing and for outputting the results
CN104601966B (en) A kind of visual analysis people flow rate statistical method and apparatus based on laser detection line
CN101999972A (en) Stereoscopic vision based auxiliary walking device for blindmen and auxiliary method thereof
CN111639709B (en) Navigation mark lamp quality intelligent recognition method based on deep learning
CN110136172B (en) Detection method for wearing of underground protective equipment of miners
CN102298693B (en) Expressway bend detection method based on computer vision
CN109902592B (en) Blind person auxiliary walking method based on deep learning
CN110119718A (en) A kind of overboard detection and Survivable Control System based on deep learning
CN110155274B (en) Underwater intelligent lifesaving system for swimming pool
CN112674998B (en) A blind traffic intersection assistance method based on fast deep neural network and mobile smart devices
CN109602585B (en) Blind guiding glasses and anti-collision early warning method thereof
CN111428680B (en) Pupil positioning method based on deep learning
CN116437538A (en) Dimming method and system for crowd gathering monitoring based on multifunctional smart light poles
CN112419745A (en) Highway group fog early warning system based on degree of depth fusion network
CN108737785B (en) Indoor automatic detection system that tumbles based on TOF 3D camera
CN113101155A (en) Traffic light intersection blind guiding method and blind guiding device based on machine vision
JP2006251596A (en) Visually impaired person support device
CN110269787A (en) A kind of blind-guiding method, device and cap
CN108563986A (en) Earthquake region electric pole posture judgment method based on wide-long shot image and system
CN107432793A (en) Safety-type Two-wheeled wheelchair
CN201870808U (en) Guide device for blind people based on stereoscopic vision
CN113343742A (en) Lane line detection method and lane line detection system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant