CN110146095B - Method and device for navigation of visually impaired people, electronic equipment and computer readable medium - Google Patents
Method and device for navigation of visually impaired people, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN110146095B CN110146095B CN201910435781.XA CN201910435781A CN110146095B CN 110146095 B CN110146095 B CN 110146095B CN 201910435781 A CN201910435781 A CN 201910435781A CN 110146095 B CN110146095 B CN 110146095B
- Authority
- CN
- China
- Prior art keywords
- visually impaired
- prompt
- road
- image
- environment image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H3/00—Appliances for aiding patients or disabled persons to walk about
- A61H3/06—Walking aids for blind persons
- A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/28—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network with correlation of data from several navigational instruments
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/3629—Guidance using speech or audio output, e.g. text-to-speech
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
- G01C21/34—Route searching; Route guidance
- G01C21/36—Input/output arrangements for on-board computers
- G01C21/3626—Details of the output of route guidance instructions
- G01C21/3652—Guidance using non-audiovisual output, e.g. tactile, haptic or electric stimuli
Abstract
The present disclosure provides a method for navigating a visually impaired person, the method comprising: an image acquisition unit carried by the visually impaired people acquires an environmental image; comparing the environment image with a preset three-dimensional map to determine the current position of the visually impaired people; and sending a navigation prompt to the visually impaired people according to the current position and the three-dimensional map. The disclosure also provides a device, an electronic device and a computer readable medium for the navigation of the visually impaired.
Description
Technical Field
The embodiment of the disclosure relates to the technical field of navigation, and in particular relates to a method and a device for visually impaired people to navigate, electronic equipment and a computer readable medium.
Background
The visually impaired people (such as the blind) face many risks and inconveniences in the outward process due to the visual defects, so that the navigation service is needed.
The navigation technology for the visually impaired people is that the visually impaired people carry navigation equipment comprising a camera and a GPS positioning unit, and images acquired through GPS positioning and the camera provide navigation service for the visually impaired people. For example, when the visually impaired people want to pass through the road, the visually impaired people can be allowed to reach a position (such as a position at one end of a crosswalk) allowing the road to pass through by GPS positioning, images of traffic lights (such as traffic lights) are collected by a camera, the images are analyzed to determine the state of the traffic lights, and when the traffic lights are in a state allowing the road (such as a state that the green lights are on), the visually impaired people are prompted to pass through the road. However, the above techniques have problems of limited application range, poor navigation accuracy, high risk, and the like.
Disclosure of Invention
The embodiment of the disclosure provides a method and a device for navigation of visually impaired people, electronic equipment and a computer readable medium.
In a first aspect, an embodiment of the present disclosure provides a method for navigating a visually impaired person, including:
an image acquisition unit carried by the visually impaired people acquires an environmental image;
comparing the environment image with a preset three-dimensional map to determine the current position of the visually impaired people;
and sending a navigation prompt to the visually impaired people according to the current position and the three-dimensional map.
In some embodiments, the issuing of the navigation prompt to the visually impaired person according to the current position and the stereoscopic map includes:
when the current position is a position ready to pass through a road and a corresponding traffic signal lamp is arranged at the position ready to pass through the road, judging whether the environment image comprises the traffic signal lamp or not;
if the environment image comprises the traffic signal lamp, judging whether the traffic signal lamp in the environment image is in a traffic-allowing state, and if the traffic signal lamp in the environment image is in the traffic-allowing state, sending a road passing prompt to the visually impaired.
In some embodiments, if the environment image does not include the traffic light, a first movement prompt is sent to the visually impaired person according to the current position and the three-dimensional map, and the first movement prompt is used for prompting the visually impaired person to move so that the image acquisition unit can acquire the environment image including the traffic light.
In some embodiments, if the traffic light in the environment image is in a passage-allowed state, the sending the notice of passage through the road to the visually impaired person comprises:
if the traffic signal lamp in the environment image is in a traffic-permitted state, determining a dangerous position where vehicles may drive on the road according to the current position and the three-dimensional map;
sending a second movement prompt to the visually impaired people according to the current position and the three-dimensional map, wherein the second movement prompt is used for prompting the visually impaired people to move so that the image acquisition unit can acquire the environmental image of the dangerous position;
acquiring an environment image of the dangerous position through the image acquisition unit carried by the visually impaired person;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, sending a road passing prompt to the visually impaired.
In some embodiments, the issuing of the navigation prompt to the visually impaired person according to the current position and the stereoscopic map includes:
when the current position is a position ready to pass through a road and the position ready to pass through the road has no corresponding traffic light, determining a dangerous position where vehicles may drive on the road according to the current position and a three-dimensional map;
sending a third movement prompt to the visually impaired people according to the current position and the three-dimensional map, wherein the third movement prompt is used for prompting the visually impaired people to move so that the image acquisition unit can acquire the environmental image of the dangerous position;
acquiring an environment image of the dangerous position through the image acquisition unit carried by the visually impaired person;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, sending a road passing prompt to the visually impaired.
In some embodiments, after the capturing the environmental image by the image capturing unit carried by the visually impaired person, the method further includes:
and sending a navigation prompt to the visually impaired people according to the environment image.
In a second aspect, an embodiment of the present disclosure provides an apparatus for navigating a visually impaired person, including:
the image acquisition unit is carried by the visually impaired and is used for acquiring an environment image;
the positioning unit is used for comparing the environment image with a preset three-dimensional map so as to determine the current position of the visually impaired person;
the prompting unit is used for sending navigation prompts to the visually impaired people;
and the position calculation unit is used for controlling the prompt unit to send out a navigation prompt according to the current position and the three-dimensional map.
In some embodiments, the location calculation unit is to:
when the current position is a position ready to pass through a road and a corresponding traffic signal lamp is arranged at the position ready to pass through the road, judging whether the environment image comprises the traffic signal lamp or not;
if the environment image comprises the traffic signal lamp, judging whether the traffic signal lamp in the environment image is in a traffic-allowing state, and if the traffic signal lamp in the environment image is in the traffic-allowing state, controlling the prompting unit to send a prompt of passing the road.
In some embodiments, the location calculation unit is to:
and if the environment image does not comprise the traffic signal lamp, controlling the prompting unit to send out a first motion prompt according to the current position and the three-dimensional map, wherein the first motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire the environment image comprising the traffic signal lamp.
In some embodiments, the location calculation unit is to:
if the traffic signal lamp in the environment image is in a traffic-permitted state, determining a dangerous position where vehicles may drive on the road according to the current position and the three-dimensional map;
controlling the prompting unit to send out a second motion prompt according to the current position and the three-dimensional map, wherein the second motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire the environmental image of the dangerous position;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and controlling the prompting unit to send out a road passing prompt if no dangerous vehicles exist.
In some embodiments, the computing unit is to:
when the current position is a position ready to pass through a road and the position ready to pass through the road has no corresponding traffic light, determining a dangerous position where vehicles may drive on the road according to the current position and a three-dimensional map;
controlling the prompting unit to send a third motion prompt according to the current position and the three-dimensional map, wherein the third motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire an environmental image of the dangerous position;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, sending a road passing prompt to the visually impaired.
In some embodiments, the apparatus further comprises:
and the image computing unit is used for controlling the prompting unit to send out a navigation prompt according to the environment image.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
one or more processors;
a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement any of the above-described methods for navigating a visually impaired person.
In a fourth aspect, the embodiments of the present disclosure provide a computer-readable medium, on which a computer program is stored, where the computer program, when executed by a processor, implements any one of the above methods for navigating a visually impaired person.
The method for navigating the visually impaired people is equivalent to determining the position of the visually impaired people through a Visual Positioning System (VPS), and the mode of positioning through comparing the acquired image with a preset map has high positioning precision, is not influenced by buildings, weather and the like, and can realize positioning in all dimensions (longitude and latitude, height and orientation), so that the position of the visually impaired people can be accurately determined under various conditions, the navigation effect is improved, and the risk is reduced.
Drawings
The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing in detail exemplary embodiments thereof with reference to the attached drawings, in which:
fig. 1 is a flowchart of a method for navigating a visually impaired person according to an embodiment of the present disclosure;
FIG. 2 is a flowchart illustrating partial steps in another method for navigating a visually impaired person according to an embodiment of the present disclosure;
FIG. 3 is a flowchart illustrating some steps in another method for navigating a visually impaired person according to an embodiment of the present disclosure;
FIG. 4 is a flowchart illustrating partial steps in another method for navigating a visually impaired person according to an embodiment of the present disclosure;
FIG. 5 is a flowchart illustrating some steps in another method for navigating a visually impaired person according to an embodiment of the present disclosure;
fig. 6 is a block diagram illustrating an apparatus for navigating a visually impaired person according to an embodiment of the present disclosure;
fig. 7 is a block diagram illustrating another apparatus for navigating a visually impaired person according to an embodiment of the present disclosure.
Detailed Description
In order to make those skilled in the art better understand the technical solutions of the present disclosure, the method and apparatus for visually impaired navigation, the electronic device, and the computer readable medium provided by the present disclosure are described in detail below with reference to the accompanying drawings.
Example embodiments will be described more fully hereinafter with reference to the accompanying drawings, but which may be embodied in different forms and should not be construed as limited to the embodiments set forth in the disclosure. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As used in this disclosure, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
When the terms "comprises" and/or "comprising … …" are used in this disclosure, the presence of stated features, integers, steps, operations, elements, and/or components are specified, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Embodiments of the present disclosure may be described with reference to plan and/or cross-sectional views in light of idealized schematic illustrations of the present disclosure. Accordingly, the example illustrations can be modified in accordance with manufacturing techniques and/or tolerances.
Embodiments of the present disclosure are not limited to the embodiments shown in the drawings, but include modifications of configurations formed based on a manufacturing process. Thus, the regions illustrated in the figures have schematic properties, and the shapes of the regions shown in the figures illustrate specific shapes of regions of elements, but are not intended to be limiting.
Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
Noun interpretation
In the embodiments of the present disclosure, the following technical terms should be understood in accordance with the following explanations, unless otherwise specified:
the visually impaired people refer to people who have certain defects in vision, so that the people cannot or cannot safely act in the road environment only depending on the vision of the people; the visually impaired persons specifically include blind persons, visually impaired persons, and persons who are in a state of blindness or amblyopia in a short time due to surgery, disease, or the like.
The Visual Positioning System (VPS) is referred to as: the method comprises the steps of collecting images of all positions of roads in advance, establishing a virtual three-dimensional map by utilizing algorithms such as SLAM, SFM, MVS and the like according to the images, calibrating information (such as traffic lights, pedestrian crossings, buildings and the like) of all scenes in the three-dimensional map, comparing the collected images with the three-dimensional map by utilizing a PnP algorithm and the like after the images are collected by an image collecting unit in the following process, and determining which part of the three-dimensional map corresponds to the image collecting unit, namely determining the position (including longitude and latitude, height, orientation and the like) of the image collecting unit.
A road, which is a road that allows vehicles (motor vehicles and/or non-motor vehicles) to pass along its length.
The passage through the road means a process of traversing the road from one side of the road to the other side of the road in the width direction of the road.
A traffic signal lamp which is arranged at the side of a road or above the road and is used for indicating whether vehicles and/or pedestrians facing the traffic signal lamp can pass or not; for example, the traffic signal light may be a traffic light, and when the red light is on, it indicates no traffic, and when the green light is on, it indicates permission to traffic. Of course, in the same position of the road, there are usually a plurality of traffic lights facing the width and length directions of the road, respectively, and when the traffic light in one direction is in a passage-permitted state, the traffic light in the other direction is usually in a passage-prohibited state. The traffic lights in the embodiments of the present disclosure are all traffic lights indicating whether the heading direction of the visually impaired person allows passage, unless otherwise specified.
The navigation technology of the visually impaired people is that the visually impaired people carry navigation equipment comprising a camera and a GPS positioning unit, and the images acquired by the GPS positioning and the camera provide navigation service for the visually impaired people. For example, when the visually impaired people want to pass through the road, the visually impaired people can be allowed to reach a position (such as a position at one end of a crosswalk) allowing the road to pass through by GPS positioning, images of traffic lights (such as traffic lights) are collected by a camera, the images are analyzed to determine the state of the traffic lights, and when the traffic lights are in a state allowing the road (such as a state that the green lights are on), the visually impaired people are prompted to pass through the road. However, the above navigation technique has at least the following problems:
(1) the GPS positioning itself has low precision, and its signal is easily affected by buildings, weather, etc., so the position of the visually impaired person often cannot be accurately determined. However, small differences in position for the navigation of the impaired may cause serious problems: for example, two positions with a difference of 0.5 m may be located on a sidewalk and a road, and if the two positions are located incorrectly, it may cause the visually impaired to walk on the road and cause serious safety problems.
(2) The above technology cannot realize effective navigation for roads without traffic lights or traffic lights with faults.
(3) Even the road has the traffic signal lamp of normal work, and GPS fixes a position accurately, above technique also can only confirm the plane position (longitude and latitude) of looking for the impaired personage, and can't confirm orientation, height etc. of looking for the impaired personage, simultaneously, look for the impaired personage itself and can't see traffic signal lamp again, the event just also can't guarantee that the camera can shoot traffic signal lamp, consequently look for the impaired personage probably need spend a large amount of time and energy go "look for" traffic signal lamp, can reduce to use and experience and cause the potential safety hazard.
Fig. 1 is a flowchart of a method for navigating a visually impaired person according to an embodiment of the present disclosure.
In a first aspect, referring to fig. 1, an embodiment of the present disclosure provides a method for navigating a visually impaired person, including:
s101, an environment image is collected through an image collecting unit carried by the visually impaired.
The environment image is acquired by an image acquisition unit carried by the visually impaired person, and is of course the image of the peripheral scenery shot from the position where the visually impaired person is located.
The image acquisition unit can be in the form of a camera, a camera and the like.
The image acquisition unit can be directly worn on the visually impaired, such as in the form of "glasses", so that the visually impaired does not need to perform special operations on the visually impaired. Of course, the image acquisition unit may be in other forms such as "hand-held".
And S102, comparing the environment image with a preset three-dimensional map to determine the current position of the visually impaired person.
Through an image analysis technology (such as a PnP algorithm), the environment image acquired above is compared with a preset three-dimensional map to determine which part of the three-dimensional map the environment image corresponds to, and further determine where and at what angle the environment image is shot on the three-dimensional map, that is, determine the current position (longitude and latitude, height, orientation) of the visually impaired person wearing the image acquisition unit.
It can be seen that the essence of this step is to determine the current position of the visually impaired person by means of a Visual Positioning System (VPS).
Of course, since the three-dimensional map usually has a large data size, it can be stored in a server (cloud), so the environment image can be sent to the server, and then the operation of this step is performed on the server side, and then the operation result is sent back to the device carried by the visually impaired.
Of course, the operation process of this step and other steps is not specifically performed, and thus the present disclosure is not limited thereto.
And S103, sending a navigation prompt to the visually impaired according to the current position and the three-dimensional map.
After the current position of the visually impaired people is determined, the conditions around the current position, such as what roads the current position is on, what buildings and what terrains the current position is around in each direction, and the like, can be determined according to the information in the three-dimensional map; therefore, according to the information, the method can determine how the visually impaired people should move currently, and sends the movement mode as a navigation prompt to the visually impaired people so as to realize navigation of the visually impaired people.
The specific content of the navigation prompt is various, as long as the navigation prompt can assist the visually impaired to move. For example, a route from the current position to the destination may be calculated according to the destination obtained in advance, and a prompt for how to proceed to the destination, such as "please keep going forward", "please turn left 10 meters ahead", etc., may be issued. For another example, the user can give a prompt of the movement mode and the attention according to the situation around the current position, such as "the road ahead is narrowed, the user should lean to the right side to avoid going up the motor vehicle lane", "the blind road ahead 10 meters disappears, the user should pay attention to", "the user has a protruding manhole cover in the front, the user should pay attention to" and the like.
The navigation prompt may be in various specific forms as long as the navigation prompt can transmit information to the visually impaired. For example, the navigation prompt may be a prompt content played by voice, or different sound effects played to represent different information, or different vibrations to represent different information, etc.
Of course, the above description is only a process of obtaining the navigation prompt according to the environment image once, and in practical applications, the above process may be repeated, that is, the image acquisition unit may acquire the environment image in real time, so as to provide a continuous navigation service for the visually impaired according to the environment image.
The method for navigating the visually impaired people is equivalent to determining the position of the visually impaired people through a Visual Positioning System (VPS), and the mode of positioning through comparing the acquired image with a preset map has high positioning precision, is not influenced by buildings, weather and the like, and can realize positioning in all dimensions (longitude and latitude, height and orientation), so that the position of the visually impaired people can be accurately determined under various conditions, the navigation effect is improved, and the risk is reduced.
In some embodiments, referring to fig. 2, the above S103 step includes:
and S1031, when the current position is the position ready to pass through the road and a corresponding traffic signal lamp is arranged at the position ready to pass through the road, judging whether the environment image comprises the traffic signal lamp.
The position for preparing the passing road is a position which is positioned at one side of the road and can pass through the road from the position; generally, the location to be passed through the road is a specific location determined according to traffic regulations, such as the location of one end of a crosswalk. Specifically, the position of the ready-to-pass road may be located at an intersection where a plurality of roads intersect (e.g., a t-intersection, an intersection, etc.), or may be located in the middle of a single road, which is not described in detail herein.
Here, the "traffic light corresponding to the position ready to pass through the road" means that the traffic light is used to indicate whether or not to allow the passage of the road from the position ready to pass through the road, and the traffic light is generally located on the opposite side of the position ready to pass through the road.
In this step, when it is determined that the visually impaired person has moved to a position ready to pass through the road (the visually impaired person may move to the position according to the previous navigation prompt), whether a traffic signal lamp corresponding to the current position exists at the current position is determined according to information in the three-dimensional map, and if so, whether a traffic signal lamp exists in the current environment image is continuously determined (i.e., whether the traffic signal lamp is shot by the image acquisition unit is determined).
Of course, if there is no corresponding traffic signal lamp at the current position, the following step S1034 (detailed description below) may be performed.
S1032, if the environment image does not include the traffic light, a first movement prompt is sent to the visually impaired people according to the current position and the three-dimensional map, and the first movement prompt is used for prompting the visually impaired people to move so that the image acquisition unit can acquire the environment image including the traffic light.
If the current position has the corresponding traffic signal lamp and the environment image does not have the traffic signal lamp, the current image acquisition unit is indicated to not shoot the traffic signal lamp, so that the visually impaired people can calculate how to move to shoot the traffic signal lamp according to the current position and the three-dimensional map, and send corresponding first movement prompts (such as prompt ' please turn left head ', please raise head ' and the like) to the visually impaired people to indicate the visually impaired people to move to the state of shooting the traffic signal lamp.
Of course, after the visually impaired person moves according to the first movement prompt, the newly captured environmental image by the image capturing unit includes the traffic light, so that the following step S1033 can be directly performed after the step S1031 is performed again.
And S1033, if the environment image comprises the traffic signal lamp, judging whether the traffic signal lamp in the environment image is in a traffic-allowing state, and if the traffic signal lamp in the environment image is in the traffic-allowing state, sending a road passing prompt to the visually impaired.
If the current environment image comprises the traffic signal lamp (can be directly comprising the traffic signal lamp, and can also be after the movement of the visually impaired people, the state (such as red light or green light) of the traffic signal lamp can be determined through an image analysis technology, and if the traffic signal lamp is in the state allowing the passage (such as green light), a prompt of passing the road is sent to the visually impaired people to inform the visually impaired people of passing the road.
Of course, if the traffic light in the current environment image is in the no-pass state (for example, the red light is on), the user can continue to wait (the user can send a waiting prompt to the visually impaired), and the user can be prompted to pass through the road until the traffic light in the environment image is in the no-pass state.
Because the three-dimensional map of the visual positioning system originally comprises the accurate position (longitude and latitude, height and orientation) of the traffic signal lamp, and the accurate position (longitude and latitude, height and orientation) of the visually impaired people (or the image acquisition unit) can be determined through the positioning, the method disclosed by the embodiment of the invention can calculate how to shoot the traffic signal lamp, and sends an accurate prompt to the visually impaired people according to the accurate position, so that the visually impaired people can easily shoot the traffic signal lamp, and can safely pass through the road, thereby avoiding the waste of time and energy and improving the safety.
In some embodiments, referring to fig. 3, the step (S1033) of issuing a prompt to the visually impaired person to pass through the road if the traffic light in the environment image is in the passage permitted state includes:
and S10331, if the traffic light in the environment image is in the traffic-allowing state, determining a dangerous position where the vehicle may come on the road according to the current position and the three-dimensional map.
When the traffic light is in the traffic-permitted state, the position (dangerous position) where vehicles may drive on the road according to the traffic rules is judged according to the current position and the three-dimensional map. The vehicles that come from the above may be vehicles that normally pass on a road (which should theoretically stop at a position ready to pass the road because traffic lights facing the vehicles should be in a no-pass state), vehicles that come from turning (for example, a part of an intersection allows a red light to turn right), and the like.
And S10332, sending a second motion prompt to the visually impaired according to the current position and the three-dimensional map, wherein the second motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire the environment image of the dangerous position.
After the dangerous position is determined, a second movement prompt (such as 'please turn the head to the left') can be sent to the visually impaired, so that the visually impaired can move, and the image acquisition unit can shoot the image of the dangerous position.
S10333, acquiring an environment image of the dangerous position through an image acquisition unit carried by the visually impaired.
When the visually impaired people move according to the instructions, images of dangerous positions can be acquired.
S10334, judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, giving a road passing prompt to the visually impaired.
By means of image analysis technology, it is determined whether there is a vehicle (dangerous vehicle) in the environment image for the dangerous location that may cause a danger to the visually impaired, such as a vehicle that is traveling at a higher speed, and a prompt to the visually impaired to pass the road is issued only when there is no dangerous vehicle.
The dangerous vehicles can be screened and identified through preset rules, for example, the distance (calculated through the size of the vehicles in the images and the like) and the speed (realized through the comparison of a plurality of images) accord with a certain relation, so that the vehicles which may collide with the visually impaired people when the visually impaired people pass through the road can be regarded as the dangerous vehicles.
According to the method, when the traffic light is in the passing-permitted state, whether vehicles possibly causing danger are actually judged according to the environment image of the position (dangerous position) where danger exists or not, and the visually impaired people can pass through the road when no danger exists, so that the safety is further improved.
Of course, it is also possible to prompt the visually impaired to pass through the road as long as the traffic light in the environment image is in the state allowing passage, without performing the above steps S10331 to S10334.
In some embodiments, referring to fig. 4, the above S103 step includes:
s1034, when the current position is the position ready to pass through the road and no corresponding traffic signal lamp is arranged at the position ready to pass through the road, determining the dangerous position where vehicles may drive on the road according to the current position and the three-dimensional map.
And S1035, sending a third movement prompt to the visually impaired according to the current position and the three-dimensional map, wherein the third movement prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire the environmental image at the dangerous position.
S1036, collecting the environment image of the dangerous position through an image collecting unit carried by the visually impaired.
S1037, judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, giving a road passing prompt to the visually impaired.
When the current position is judged to have no corresponding traffic signal lamp according to the current position and the three-dimensional map, whether the road passes through can be determined only by observing the actual road condition. Specifically, in a manner similar to the above steps S10331 to S10334, it may be determined what position of the road may be driven by the vehicle (i.e., the dangerous position), and prompt the visually impaired to move to acquire the environment image of the position, and finally, it may be determined whether the road can be safely passed through by the environment image of the dangerous position, so as to improve the safety, and the method of the embodiment of the present disclosure may be used in the case that there is no traffic light (or the traffic light is damaged).
In some embodiments, referring to fig. 5, after the step S102, the method further includes:
and S104, sending a navigation prompt to the visually impaired according to the environment image.
After the environment image is obtained, the environment image itself can be analyzed through an image analysis technology without combining a three-dimensional map for the moment, factors which may influence the movement of the visually impaired people are determined, and a navigation prompt is sent to the visually impaired people according to the factors, such as 'an unknown obstacle exists in front of 5 meters, the visually impaired people need to walk around to the left', 'the visually impaired people are too close to the pedestrian in front, and the forward speed is required to be slowed down', and the like.
Therefore, objects (such as pedestrians, vehicles, temporary obstacles and the like) which do not exist in the three-dimensional map can be found in the above mode, and therefore better navigation service is provided for the visually impaired.
Fig. 6 is a block diagram of a device for navigating the visually impaired according to an embodiment of the present disclosure.
In a second aspect, referring to fig. 6, an embodiment of the present disclosure provides an apparatus for navigating a visually impaired person, including:
the image acquisition unit is carried by the visually impaired and is used for acquiring an environment image;
the positioning unit is used for comparing the environment image with a preset three-dimensional map so as to determine the current position of the visually impaired person;
the prompting unit is used for sending navigation prompts to the visually impaired;
and the position calculating unit is used for controlling the prompting unit to send out a navigation prompt according to the current position and the three-dimensional map.
The image acquisition unit and the prompt unit are carried by the visually impaired people necessarily because images need to be acquired and prompts need to be sent out. Specifically, the image acquisition unit can be a camera, and the like, and can be in the form of glasses; and the prompting unit can be in the form of a sound production unit, a vibration unit and the like.
As mentioned above, part of the calculation process of the embodiment of the present disclosure may be performed in the server (cloud). Therefore, in the device for navigating the visually impaired people according to the embodiment of the present disclosure, some units may also be located at the server side, and of course, if some units are located at the server side, a unit (such as a wireless communication unit) for sending and receiving information should be included in the device of the cuff of the visually impaired people and the device at the server side, so as to implement interaction between the devices at both ends, which is not described in detail herein.
In some embodiments, the location calculation unit is to:
when the current position is a position ready to pass through a road and a corresponding traffic signal lamp is arranged at the position ready to pass through the road, judging whether the environment image comprises the traffic signal lamp or not;
and if the environment image comprises the traffic signal lamp, judging whether the traffic signal lamp in the environment image is in a traffic-allowing state, and if the traffic signal lamp in the environment image is in the traffic-allowing state, controlling a prompting unit to send out a prompt of passing the road.
In some embodiments, the location calculation unit is to:
if the environment image does not include the traffic signal lamp, the prompting unit is controlled to send out a first movement prompt according to the current position and the three-dimensional map, and the first movement prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire the environment image including the traffic signal lamp.
In some embodiments, the location calculation unit is to:
if the traffic light in the environment image is in a traffic-permitted state, determining a dangerous position where vehicles may come on the road according to the current position and the three-dimensional map;
controlling the prompting unit to send out a second motion prompt according to the current position and the three-dimensional map, wherein the second motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire an environment image of a dangerous position;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and controlling a prompting unit to give a road passing prompt if no dangerous vehicles exist.
In some embodiments, the computing unit is to:
when the current position is the position ready to pass through the road and the position ready to pass through the road has no corresponding traffic signal lamp, determining the dangerous position of the road where vehicles may drive according to the current position and the three-dimensional map;
controlling a prompting unit to send a third motion prompt according to the current position and the three-dimensional map, wherein the third motion prompt is used for prompting the visually impaired to move so that an image acquisition unit can acquire an environment image of a dangerous position;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicle exists, sending a road passing prompt to the visually impaired.
In some embodiments, referring to fig. 7, the apparatus further comprises:
and the image computing unit is used for controlling the prompting unit to send out a navigation prompt according to the environment image.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
one or more processors;
a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement any of the above-described methods for navigating a visually impaired person.
In a fourth aspect, the embodiments of the present disclosure provide a computer-readable medium, on which a computer program is stored, where the computer program, when executed by a processor, implements any one of the above methods for navigating a visually impaired person.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
The present disclosure has disclosed example embodiments and, although specific terms are employed, they are used and should be interpreted in a generic and descriptive sense only and not for purposes of limitation. In some instances, features, characteristics and/or elements described in connection with a particular embodiment may be used alone or in combination with features, characteristics and/or elements described in connection with other embodiments, unless expressly stated otherwise, as would be apparent to one skilled in the art. Accordingly, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the disclosure as set forth in the appended claims.
Claims (10)
1. A method of navigating a visually impaired person, comprising:
an image acquisition unit carried by the visually impaired people acquires an environmental image;
comparing the environment image with a preset three-dimensional map to determine the current position of the visually impaired people;
sending a navigation prompt to the visually impaired people according to the current position and the three-dimensional map;
the sending of the navigation prompt to the visually impaired comprises the following steps of according to the current position and the three-dimensional map:
when the current position is a position ready to pass through a road and a corresponding traffic signal lamp is arranged at the position ready to pass through the road, judging whether the environment image comprises the traffic signal lamp or not;
if the environment image comprises the traffic signal lamp, judging whether the traffic signal lamp in the environment image is in a traffic-allowing state, and if the traffic signal lamp in the environment image is in the traffic-allowing state, sending a road passing prompt to the visually impaired;
if the environment image does not comprise the traffic signal lamp, a first motion prompt is sent to the visually impaired people according to the current position and the three-dimensional map, and the first motion prompt is used for prompting the visually impaired people to move so that the image acquisition unit can acquire the environment image comprising the traffic signal lamp.
2. The method of claim 1, wherein issuing a prompt to the visually impaired to pass the road if the traffic light in the environmental image is in a passage-allowed state comprises:
if the traffic signal lamp in the environment image is in a traffic-permitted state, determining a dangerous position where vehicles may drive on the road according to the current position and the three-dimensional map;
sending a second movement prompt to the visually impaired people according to the current position and the three-dimensional map, wherein the second movement prompt is used for prompting the visually impaired people to move so that the image acquisition unit can acquire the environmental image of the dangerous position;
acquiring an environment image of the dangerous position through the image acquisition unit carried by the visually impaired person;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, sending a road passing prompt to the visually impaired.
3. The method of claim 1, wherein said issuing navigation prompts to the visually impaired according to the current location and a relief map comprises:
when the current position is a position ready to pass through a road and the position ready to pass through the road has no corresponding traffic light, determining a dangerous position where vehicles may drive on the road according to the current position and a three-dimensional map;
sending a third movement prompt to the visually impaired people according to the current position and the three-dimensional map, wherein the third movement prompt is used for prompting the visually impaired people to move so that the image acquisition unit can acquire the environmental image of the dangerous position;
acquiring an environment image of the dangerous position through the image acquisition unit carried by the visually impaired person;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, sending a road passing prompt to the visually impaired.
4. The method of claim 1, wherein after the capturing the environmental image by the image capture unit carried by the visually impaired person, further comprising:
and sending a navigation prompt to the visually impaired people according to the environment image.
5. An apparatus for navigating a visually impaired person, comprising:
the image acquisition unit is carried by the visually impaired and is used for acquiring an environment image;
the positioning unit is used for comparing the environment image with a preset three-dimensional map so as to determine the current position of the visually impaired person;
the prompting unit is used for sending navigation prompts to the visually impaired people;
the position calculation unit is used for controlling the prompt unit to send out a navigation prompt according to the current position and the three-dimensional map;
the position calculation unit is configured to:
when the current position is a position ready to pass through a road and a corresponding traffic signal lamp is arranged at the position ready to pass through the road, judging whether the environment image comprises the traffic signal lamp or not;
if the environment image comprises the traffic signal lamp, judging whether the traffic signal lamp in the environment image is in a traffic-allowing state, and if the traffic signal lamp in the environment image is in the traffic-allowing state, controlling the prompting unit to send a prompt of passing a road;
and if the environment image does not comprise the traffic signal lamp, controlling the prompting unit to send out a first motion prompt according to the current position and the three-dimensional map, wherein the first motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire the environment image comprising the traffic signal lamp.
6. The apparatus of claim 5, wherein the location calculation unit is to:
if the traffic signal lamp in the environment image is in a traffic-permitted state, determining a dangerous position where vehicles may drive on the road according to the current position and the three-dimensional map;
controlling the prompting unit to send out a second motion prompt according to the current position and the three-dimensional map, wherein the second motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire the environmental image of the dangerous position;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and controlling the prompting unit to send out a road passing prompt if no dangerous vehicles exist.
7. The apparatus of claim 5, wherein the computing unit is to:
when the current position is a position ready to pass through a road and the position ready to pass through the road has no corresponding traffic light, determining a dangerous position where vehicles may drive on the road according to the current position and a three-dimensional map;
controlling the prompting unit to send a third motion prompt according to the current position and the three-dimensional map, wherein the third motion prompt is used for prompting the visually impaired to move so that the image acquisition unit can acquire an environmental image of the dangerous position;
and judging whether dangerous vehicles exist in the environment image of the dangerous position, and if no dangerous vehicles exist, sending a road passing prompt to the visually impaired.
8. The apparatus of claim 5, further comprising:
and the image computing unit is used for controlling the prompting unit to send out a navigation prompt according to the environment image.
9. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon which, when executed by the one or more processors, cause the one or more processors to implement the method for navigating a visually impaired person according to any one of claims 1 to 4.
10. A computer-readable medium, on which a computer program is stored which, when being executed by a processor, carries out the method of navigating a visually impaired person according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910435781.XA CN110146095B (en) | 2019-05-23 | 2019-05-23 | Method and device for navigation of visually impaired people, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910435781.XA CN110146095B (en) | 2019-05-23 | 2019-05-23 | Method and device for navigation of visually impaired people, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110146095A CN110146095A (en) | 2019-08-20 |
| CN110146095B true CN110146095B (en) | 2021-07-20 |
Family
ID=67592745
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910435781.XA Active CN110146095B (en) | 2019-05-23 | 2019-05-23 | Method and device for navigation of visually impaired people, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110146095B (en) |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101999972B (en) * | 2010-11-24 | 2013-07-03 | 上海理工大学 | Stereoscopic vision based auxiliary walking device for blindmen and auxiliary method thereof |
| KR20130055970A (en) * | 2011-11-21 | 2013-05-29 | 현대모비스 주식회사 | Method and avn system for guiding vehicle's running direction using right and left differentiation of sound volume |
| CN104700613B (en) * | 2015-02-27 | 2017-12-19 | 惠州Tcl移动通信有限公司 | Method, mobile terminal and the road instruction equipment that guiding blind person goes across the road |
| CN105030491B (en) * | 2015-07-17 | 2017-09-19 | 上海斐讯数据通信技术有限公司 | A kind of blind-guiding method and blind guiding system |
| CN105686935B (en) * | 2016-01-08 | 2018-06-01 | 中国石油大学(华东) | A kind of intelligent blind-guiding method |
| CN105832501A (en) * | 2016-03-23 | 2016-08-10 | 京东方科技集团股份有限公司 | Blind person navigation method and blind person navigation equipment |
| CN106859929B (en) * | 2017-01-25 | 2019-11-22 | 上海集成电路研发中心有限公司 | A kind of Multifunctional blind person guiding instrument based on binocular vision |
| JP6923246B2 (en) * | 2017-02-07 | 2021-08-18 | Ｎｅｃソリューションイノベータ株式会社 | Crossing support systems, crossing support methods, programs, and recording media |
| CN107328424B (en) * | 2017-07-12 | 2020-12-11 | 三星电子（中国）研发中心 | Navigation method and device |
| CN107389086B (en) * | 2017-07-17 | 2020-07-14 | 惠州Tcl移动通信有限公司 | Full-visual navigation positioning method, intelligent terminal and storage device |
| CN107506760B (en) * | 2017-08-04 | 2020-06-05 | 西南大学 | Traffic signal detection method and system based on GPS positioning and visual image processing |
| CN207163466U (en) * | 2017-09-06 | 2018-03-30 | 钟泳炽 | One visually impaired people's navigation system |
| CN108564785A (en) * | 2018-03-27 | 2018-09-21 | 合肥龙图腾信息技术有限公司 | A kind of navigation system for blinds with safety traffic light intersection function |
-
2019
- 2019-05-23 CN CN201910435781.XA patent/CN110146095B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110146095A (en) | 2019-08-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7302934B2 (en) | System and method for anonymizing navigation information | |
| US11402851B2 (en) | Systems and methods for detecting low-height objects in a roadway | |
| JP7329298B2 (en) | Vehicle navigation based on detected barriers | |
| CN112629551B (en) | Navigation system with imposed responsibility constraints | |
| CN107111742B (en) | Identification and prediction of lane restrictions and construction areas in navigation | |
| CN111027420B (en) | System and method for mimicking a lead vehicle | |
| JP7194130B2 (en) | A method and apparatus for detecting emergency vehicles in real time and planning driving routes to deal with situations expected to be caused by emergency vehicles. | |
| US10565863B1 (en) | Method and device for providing advanced pedestrian assistance system to protect pedestrian preoccupied with smartphone | |
| KR20200131832A (en) | Information processing devices, mobile devices and methods, and programs | |
| JP2023504604A (en) | System and method for selectively decelerating a vehicle | |
| CN110146095B (en) | Method and device for navigation of visually impaired people, electronic equipment and computer readable medium | |
| CN114333339B (en) | Deep neural network functional module de-duplication method | |
| JP2020126611A (en) | Method and device for learning occurrence of ldw alarm with reference to information on driving condition used for adas, v2x, or driver safety required for satisfying level 4 and level 5 of autonomous vehicle | |
| JP2019163969A (en) | On-vehicle device and image display method | |
| CN117572343A (en) | Short-distance navigation method and device for visually impaired user, electronic equipment and storage medium | |
| KR20230071920A (en) | Vehicle control system and navigating method using vehicle control system | |
| KR20230071610A (en) | Vehicle control system and navigating method using vehicle control system | |
| KR20230071921A (en) | Vehicle control system and navigating method using vehicle control system | |
| KR20230071609A (en) | Vehicle control system and navigating method using vehicle control system | |
| KR20230071924A (en) | Vehicle control system and navigating method using vehicle control system | |
| KR20230071615A (en) | Vehicle control system and navigating method using vehicle control system | |
| KR20230071611A (en) | Vehicle control system and navigating method using vehicle control system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| EE01 | Entry into force of recordation of patent licensing contract |
Application publication date: 20190820 Assignee: Beijing Intellectual Property Management Co.,Ltd. Assignor: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY Co.,Ltd. Contract record no.: X2023110000093 Denomination of invention: Navigation methods and devices for visually impaired individuals, electronic devices, and computer-readable media Granted publication date: 20210720 License type: Common License Record date: 20230818 |
|
| EE01 | Entry into force of recordation of patent licensing contract |