CN106389078A - Intelligent blind guiding glass system and blind guiding method thereof - Google Patents
Intelligent blind guiding glass system and blind guiding method thereof Download PDFInfo
- Publication number
- CN106389078A CN106389078A CN201611039189.0A CN201611039189A CN106389078A CN 106389078 A CN106389078 A CN 106389078A CN 201611039189 A CN201611039189 A CN 201611039189A CN 106389078 A CN106389078 A CN 106389078A
- Authority
- CN
- China
- Prior art keywords
- module
- image
- central control
- control module
- cloud server
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H3/00—Appliances for aiding patients or disabled persons to walk about
- A61H3/06—Walking aids for blind persons
- A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H3/00—Appliances for aiding patients or disabled persons to walk about
- A61H3/06—Walking aids for blind persons
- A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
- A61H2003/063—Walking aids for blind persons with electronic detecting or guiding means with tactile perception
Abstract
The invention provides an intelligent blind guiding glass system and a blind guiding method thereof. The intelligent blind guiding glass system comprises an image collecting module, a voice processing module, a 4G communication module, a central control module and a cloud server. The image collecting module collects an image of the surrounding of a user and uploads the obtained image to the central control module, the voice processing module is connected with the central control module, and the 4G communication module establishes real-time intercommunication between the glass end and the cloud server responsible for recognizing and distance measuring and achieves the voice communication function; the central control module carries out recognition and distance measurement on common objects through image template matching and a disparity map, and if recognition cannot be carried out, the result is sent to the cloud server through the 4G communication module, and the cloud server builds an image analysis system by establishing a software platform on the cloud server; an image learning database is established through a deep learning neural network, and the information of the distance and the class of a barrier is obtained from the uploaded image and fed back to the central control module. In this way, the problems that existing intelligent blind guiding glasses are single in function and low in intelligent level can be solved. The intelligent blind guiding glass system and the blind guiding method belong to the field of intelligent blind guiding.
Description
Technical field
The present invention relates to an intelligent blind guiding system and method method, belong to guide field.
Background technology
China is the most country of world blind person, there are about 12,000,000, accounts for the 18% of whole world blind person's mouth, as social groups
In special population, they live in boundless dark throughout one's life, therefore usually can run into various difficult problems, in the market one
A little most structures of guide glasses are simple and single function (can only simply point out front to have barrier) is although some products use
Convenient, but auxiliaring effect inconspicuous, and, friend blind person can encounter problems when using, and such as road conditions are bad, hollow
Injustice, there is barrier of suspension etc. in front, and common glasses cannot accurately be verified, and relatives are not easy the problems such as find, this
A little problems are insurmountable.The obstacle detection function of existing guide product, is limited only to utilize ultrasonic listening at present
Technology is detected to the distance of barrier it is impossible to be accurately positioned to barrier place orientation, and can only be to single barrier
Hinder thing to be detected, such as in the detection doing more physical exercises barrier, be merely capable of detecting from the closest barrier of user,
Hence in so that the guiding function practicality of product is greatly lowered.
Actually with regard to smart guide glasses, carried out research in international and national Dou You team and individual, but all the time
Rest on performance detection and feasibility study stage, the mass production of product is not carried out to it so far, especially at home,
Research and development to intelligent blind-guiding equipment have even more just been in the starting stage, away from mass product, are commercialized also very long one section
Distance.
Although nowadays the existing patent commonly turning glasses in market has, ununified professional standard, especially
Be Multifunctional blind person guiding glasses exclusively for friend's blind person research and design patent even more few, and most products,
There are a lot of defects and problem when actually used.
At present, commercially also there is no the smart guide glasses of a practical application, great majority are also merely resting on development
Or try on improving, and single function, level of intelligence are relatively low.Google issued a " expanding reality " eye in 2012
Mirror, it possesses the function the same with smart mobile phone, can be taken pictures by sound control, video calling, and surfs the web, processes
Word message and Email etc., but these functions also simply perfect personage's exploitation design towards vision, and this is also on market
The primary version technique that unique examination is issued, some glasses for guiding blind products other all also rest in concept and research and development.So, compel
It is essential and want a kind of convenient, efficient smart guide glasses exclusively for visually impaired person development, and recognition type scheme of going forward one by one
Orientation, range finding and object discriminant function, and have that positioning etc. is one-touch to complete function, when can carry out with backstage Cloud Server
When interaction communication system, also can be that later stage constantly improve, Function Extension provide wide headspace.
Content of the invention
It is an object of the invention to：There is provided a kind of smart guide glasses system and its blind-guiding method, to solve existing intelligence
The relatively low problem of glasses for guiding blind single function, level of intelligence.
For solving the above problems, provide a kind of smart guide glasses system, including：
Image capture module, the ambient image around collection user, and the image of acquisition is uploaded to central authorities' control mould
Block；
Speech processing module, is connected with central control module, for the man-machine interaction of user and various functions intermodule；
4G communication module, is glasses end and the Cloud Server being responsible for identification range finding is set up and is in communication with each other in real time, and realizes language
Sound call function；
Central control module, the data that each functional unit is collected and information carry out Macro or mass analysis, and according to data
The scene being reflected, sends corresponding control instruction to each functional module；Familiar object is mated by image template and regards
Difference figure is identified and finds range；As distinguished, 4G communication module is recycled to send to Cloud Server；
Cloud Server, sets up image analysis system by building software platform on Cloud Server, using deep learning god
Through network, set up image study database, using deep learning and binocular location algorithm, obtain barrier from the image uploading
Distance and classification information, and feedback of the information is returned central control module；
USB diverter module, for image capture module is connected with central control module with satellite positioning module, realizes letter
Breath alternating transmission；
Satellite positioning module, is connected with central control module, and for satellite fix, GPS can also be obtained by 4G communication module
Latitude and longitude information sent to PC server end and the reception server feedback information by Internet；
The present invention also provides a kind of smart guide glasses blind-guiding method, including：
Ambient image around user is gathered by image capture module, the image of acquisition is passed using local-server
Enter identifying schemes：First by central control module, familiar object is mated by image template and disparity map is identified and finds range；
As distinguished, the network communicating function of 4G communication module is recycled to send to Cloud Server, Cloud Server utilizes deep learning
With binocular range finding scheduling algorithm, obtain obstacle distance and classification information from the image uploading, and information is sent out with text formatting
Send central control module back to, then voice messaging is converted it into by speech processing module, blind person is informed by earphone.Additionally, working as
When user is badly in need of informing kith and kin oneself current location, satellite positioning module can place latitude and longitude information has clothes at present by user
Business device is converted into positional information, is sent to kith and kin's mobile phone through 4G communication module, so that kith and kin's missing in the way of note.
Described local-server goes forward one by one identifying schemes, that is, to some common barriers first on the central control module of glasses end
The picture hindering thing carries out feature extraction and carries out machine learning setting up matching template taxonomy database, after identification distance measurement function is opened,
The central control module first picture decompression to pretreatment, is carried out to object and background point using Research on threshold selection in picture
Cut, then with Boost method, feature extraction is carried out to the target image after background segment, the target signature getting passes through SVM method
Mated with setting up familiar object template database, thus obtaining the recognition result of familiar object；If matching result similarity
Less than identification ratio, central control module can send Cloud Server pretreated image to by 4G communication module to be known
Not, Cloud Server carries out study identification by the database that the deep learning set up was trained, and is drawn according to highest similarity
Discrimination results.Local identification, distance measuring method can save SRT, and server mode may insure not familiar object
Accurately identify, the coordinated operation of two ways ensure that real-time, accuracy and popularity of image object identification etc..
Image capture module is the binocular camera of simulation human eye, is connected with central control module through USB diverter module, with
The principle of simulation mankind's right and left eyes visual imaging obtains two width images of same scenery, and the two width pictures getting are to be taken the photograph by binocular
As the jpeg form of head hardware-compressed, then it is transferred to central control module, by the two dimensional image acquired in two cameras
Calculate the three-dimensional information of scenery in conjunction with neural network algorithm and extract main information in picture, image is drawn by BM algorithm
Parallax, it compares with the binocular vision of the mankind alike in principle.Because binocular camera shooting can provide the space of observed object
Steric information, thus have a wide range of applications in field of machine vision.Meanwhile, the various signal transacting that binocular camera shooting need to be carried out relate to
And the various aspects of image processing techniques, comprehensive very strong.At present, conventional binocular machine vision system is substantially to adopt
Two optical axis are parallel or the collection to complete image for the video camera of fixed angle.This mode has that structure is simple, measuring speed
The advantages of high precision that hurry up, measure and mate.But system is before carrying out the measurements, need video camera is carried out strict parameter mark
Fixed.
The general principle of binocular range finding is as follows：
Setting the distance between human body and object is Z, the focal length f of camera, the distance between camera lens of two cameras
For T, object on the two sensors imaging xl and xr apart from d, be variable, have：
D=xl-xr
It is thus known that the value of d, you can Z is obtained according to similar triangle theory；
Binocular range finding is realized on Opencv, mainly comprises the following steps：
1. binocular correction and demarcation, obtain the parameter matrix of camera, demarcate and use MATLAB calibration tool；
2. Stereo matching, obtains disparity map；
3. adopt BM algorithm generate disparity map, if left and right match point is denser, match point is many, the image obtaining with former
Figure similarity is high, if match point ratio is sparse, the point obtaining is low with artwork similarity；
4. draw range finding；
5. the disparity map generating is input to reprojectImageTo3D () function, generates 3D point cloud, protect in 3D point cloud
There is a three-dimensional coordinate of 2D image, then reads the value of the z-axis in the three-dimensional coordinate of every two field picture, just drawn range data；
6. interpretation of result.
Target obstacle orientation discrimination：The Target Photo of background under Opencv environment, is obtained by threshold selection method,
And coordinate is set up to this picture whole region carry out region division, it is divided into three regions, respectively left front, front, right front；
Meanwhile, on the basis of the Target Photo removing background, target corresponding profile block diagram is generated according to target concrete shape, further according to
This block diagram, calculates the geometric center point of objective contour block diagram using geometrical principle；After the completion of central point and region division, according to
Objective contour central point to draw this target currently with respect to the concrete side of camera positioned at the concrete region of region division in figure
Position.
Binocular camera and satellite positioning module are connected with central control module through USB diverter module respectively, USB divergent die
Block is FE1.1s, and FE1.1s is a highly integrated, high-quality, high-performance, low energy consumption, simultaneously or USB2.0 4 port line concentrations at a high speed
Low cost solution.It adopts single exchange converter (Single Transaction Translator) (STT) to build
To obtain more benefits.Six rather than two aperiodic translation caches are to reduce potential Communication Jamming.Whole design
Based on state machine control, to reduce the time delay of response；Not using microcontroller in this chip.For ensureing high-quality,
Whole chip passes through to test scan chain (Test Scan Chain) even if under (480MHz) pattern at a high speed, making all of logic
Element is fully tested before shipment.Particularly being using all of Built In Self Test (Build-In-Self-Test) type order
High speed, at full speed, and low-speed mode AFE(analog front end) port (AFE) is also such in the packaging and testing stage.The reality of low energy consumption
It is now by using 0.18m technology and integrated power supply/clock control mechanism.Most of pin does not need timing, unless quilt
Use；
Speech processing module, this module includes pronounciation processing chip SYN6288, major function be exactly realize user with each
Plant the man-machine interaction between functional module, when user is less than threshold value of warning apart from the distance of barrier, central control module meeting
Send broadcast command to this module, the data obtaining then is processed by this module, and entered by outside speaker or ear speaker device
Row is reported, and informs classification, the azimuth-range of user's preceding object thing, for example, " has vehicle at 2.3 meters of right front " so that making
User can carry out avoidance adjustment.When user wonders current location, navigation button can be triggered, be worked as by voice broadcast
Front geographical location information.
Global position system is global positioning system Global Positioning System it is simply that using satellite to certain
The technology that thing is accurately positioned.Can ensure that at any time, on the earth, any point can observe 4 simultaneously and defend
Star, to realize the functions such as navigation, positioning, time service.The round-the-clock, high accuracy having and the feature of automatic measurement, as advanced
Measurement means and new productivity, incorporated the development of the national economy, national defense construction and social development each application neck
Domain.The operation principle of global position system is to measure the satellite of known location the distance between to receiver user, then comprehensive
The data closing multi-satellite knows the particular location of receiver.
GPS application in this technique, mainly makes glasses have accurate satellite positioning functions, its be achieved in that by
The client that glasses communicate as Socket, the latitude and longitude information that GPS is obtained utilizes 4G communication module to pass through ICP/IP protocol
Pass to PC server end on internet, then the latitude and longitude information obtaining is passed through Baidu map by PC server again
API carries out mating of longitude and latitude and positional information, thus learning user position, position is believed by last PC server end again
Breath sends back central control module by internet.
As user it should be understood that during present position or wanting to allow household and friend will appreciate that oneself is currently located
During particular geographic location, trigger switch, 4G communication module positional information will send close relative people's mobile phone in the way of note.
The realization of network communicating function, make use of the wireless broadband network access function of 4G communication module, the work(of this module
The foundation of communication link between glasses end and each function Cloud Server platform can be responsible for, one of them, be by glasses end and to be responsible for
The Cloud Server of identification range finding is set up and is in communication with each other in real time, and realizes voice call function, is made with facilitating its household to understand in time
User's current physical condition；Second, wireless communication module not only achieves the function of contacting with household, also achieve and GPS is obtained
The latitude and longitude information taking sent to PC server end by Internet and the reception server feedback information network communicating function,
And this function is also vital link in satellite fix link.
4G communication module is connected with central control module by USB interface, and USB main line bus signals are 4 modules and system
Between communication interface.Driven according to the ECM mouth carrying of Linux3.2.0 kernel and ME3760_V2 module is loaded, make
It is that the USB port of ME3760_V2 module is attached, under linux, the ECM mouth of ME3760_V2 module is mapped to 5
Interface：ECM, AT, Modem, Log (are hereafter discussed in detail), wherein " " fall within the part of ECM mouth, for preventing ECM
Function is capped, and should be filtered when carrying out USB initialization of (a) serial ports, and loads the PPP of linux kernel and drive so as to remaining
Interface initialization is USB serial equipment, finally uses PPP instrument dial-up connection 4G network.
Technology realizes step：
1. the USB port of ME3760_V2 module is mapped to 5 interfaces under linux：ECM ,/, AT, Modem and Log；
2. kernel modifications, add the driving of ME3760_V2 module, such as on the basis of original linux version：Interpolation sets
Standby information, USB serial port drive filter ECM interface；
3. kernel compiling；
4. equipment loads, and the ECM mouth of ME3760_V2 module is mapped to network interface eth0, wherein AT/Modem/ in systems
Log mouth is mapped as ttyUSB0-ttyUSB2 in systems；
5. set up 4G to connect, using AT instruction, dialled using PPP, and the IP of static state setting eth0 and gateway DNS or make
With PPP dialing, just 4G network can be used.
Central control module, this module mainly realize data that whole glasses various pieces functional unit is collected and
Information carries out Macro or mass analysis, and the scene being reflected according to data, sends corresponding control instruction to each functional module.
In order that modules operational efficiency is higher, performance is more stable, real-time performance is higher, team takes exclusively for core board
Carry LINUX PDA operating system, and its major function in the design has been embodied in following several aspects:
In the control of the IMAQ of binocular camera, use program setting one in advance in TQ210_COREB core board
Individual picture-taken frequency, and in real time by the compression of images having gathered and be sent to background server, core board constantly receives backstage
The image analysis result information that server passes over.
Go forward one by one in local-server the glasses end of identifying schemes, be equipped with simplifying the A8 of (SuSE) Linux OS first
On, using Opencv platform, familiar object is carried out with Boost method with feature extraction, and machine learning is used to the feature extracted
Set up matching template database.After identification distance measurement function is opened, the first picture decompression to pretreatment of glasses end, right in picture
Object is split with background, then carries out feature extraction to the target image after background segment, and the target signature getting is sharp again
Mated with setting up familiar object template database with SVM method, thus obtaining the recognition result of familiar object；If coupling knot
Fruit similarity is less than identification ratio, and glasses end can send Cloud Server pretreated image to by 4G network and be known
Not.
In the control to speech processing module, it is provided with threshold value of warning in background server analysis program in advance, when
When the object distance detecting is less than threshold value of warning, the text message receiving is obtained after voice module process by core board
Various functions sound bite, pass to user in the form of outside speaker or earphone, when detecting object distance is more than
During threshold value of warning, stop reporting.
When user is in and gets lost or be badly in need of the help obtaining friend relatives, pressed by touching the transmission of opsition dependent information
Key, after master control system obtains trigger, can adopt interrupt mode, control satellite positioning module to determine the longitude and latitude of current location
Then latitude and longitude information is converted to specific geographical location information by central control module by information again, finally by 4G communication
Module sends it to the household's phone number prestoring, and to facilitate household to understand its present position, carries out missing.
Deep learning principle：Deep learning is in tranining database, it is desirable to provide powerful computing capability, and needs to deposit
Store up substantial amounts of training data, in view of Cloud Server has above advantage and characteristic, decide using Cloud Server as image recognition
The software platform of processing system.The concept of deep learning comes from the research of artificial neural network.Multilayer perceptron containing many hidden layers
It is exactly a kind of deep learning structure.Deep learning pass through to combine low-level feature formed more abstract high-rise represent attribute classification or
Feature, is represented with the distributed nature finding data.Deep learning is one of machine learning research new field, its motivation
It is to set up, simulation human brain is analyzed the neutral net that learns, it imitates the mechanism of human brain explaining data, such as image,
Sound and text.
Compared with prior art, the present invention is convenient, the efficient intelligent blind-guiding developed exclusively for visually impaired person
Glasses, have orientation, range finding and the object discriminant function of recognition type scheme of going forward one by one, and have that positioning etc. is one-touch to complete work(
Can, the communication system that constantly can be interacted with backstage Cloud Server, also can be wide for later stage constantly improve, Function Extension offer
Headspace；Using the cutting edge technology such as machine vision, Digital Image Processing and identification, deep learning, computer network, realize
To the classification of wearer's preceding object thing, orientation, the comprehensive survey of distance and prompting, satellite fix, voice broadcast, 4G network
The function such as communication and one-touch note contact household, local identification, distance measuring method can save SRT, server side
Formula may insure accurately identifying of not familiar object, the coordinated operation of two ways ensure that the real-time of image object identification,
Accuracy and popularity etc., have very wide application prospect.
Brief description
Fig. 1 is smart guide glasses hardware system block diagram；
Fig. 2 is smart guide glasses system operation flow chart；
Fig. 3 is binocular range measurement principle figure；
Fig. 4 is speech processing module control block diagram；
Fig. 5 is the positioning flow figure of satellite positioning module；
Fig. 6 is glasses end obstacle recognition, azimuth-range realization procedure chart；
Fig. 7 is the deep learning illustraton of model containing multiple hidden layers；
Fig. 8 is the training mechanism choice of layer-wise.
Specific embodiment
For making the object, technical solutions and advantages of the present invention clearer, the present invention will be made further detailed below
Description,
Embodiment：
See figures.1.and.2, the present embodiment provides a kind of smart guide glasses system, including：
Image capture module 1, the ambient image around collection user, and the image of acquisition is uploaded to central authorities' control mould
Block 4；
Speech processing module 2, is connected with central control module 4, for the man-machine friendship of user and various functions intermodule
Mutually；
4G communication module 3, is glasses end and the Cloud Server 5 being responsible for identification range finding is set up and is in communication with each other in real time, and realizes
Voice call function；
Central control module 4, the data that each functional unit is collected and information carry out Macro or mass analysis, and according to data
The scene being reflected, sends corresponding control instruction to each functional module；Familiar object is mated by image template and regards
Difference figure is identified and finds range；As distinguished, 4G communication module 3 is recycled to send to Cloud Server 5；
Cloud Server 5, sets up image analysis system by building software platform on Cloud Server, using deep learning god
Through network, set up image study database, using deep learning and binocular location algorithm, obtain barrier from the image uploading
Distance and classification information, and feedback of the information is returned central control module 4；
USB diverter module 6, for image capture module 1 is connected with central control module 4 with satellite positioning module 4, real
Existing information exchange transmission；
Satellite positioning module 7, is connected with central control module 4, and for satellite fix, GPS can also be obtained by 4G communication module 3
The latitude and longitude information taking is sent to PC server end and the reception server feedback information by Internet；
The blind-guiding method of above-mentioned smart guide glasses is as follows：
Ambient image around user is gathered by image capture module 1, the image of acquisition is passed using local-server
Enter identifying schemes：First by central control module 4, familiar object is mated by image template and disparity map is identified and finds range；
As distinguished, the network communicating function of 4G communication module 3 is recycled to send to Cloud Server 5, Cloud Server 5 utilizes depth
Practise and binocular range finding scheduling algorithm, from the image uploading, obtain obstacle distance and classification information, and by information with text formatting
Send back central control module 4, then voice messaging is converted it into by speech processing module 2, blind person is informed by earphone.This
Outward, when user is badly in need of informing kith and kin oneself current location, satellite positioning module 7 can be by user's place longitude and latitude letter at present
Breath has server to be converted into positional information, is sent to kith and kin's mobile phone through 4G communication module 3, so that kith and kin seek in the way of note
People.
Locally-server goes forward one by one identifying schemes, that is, to some common impairments things first on glasses end central control module 4
Picture carry out feature extraction and carry out machine learning setting up matching template taxonomy database, after identification distance measurement function is opened, central authorities
The control module 4 first picture decompression to pretreatment, is split with background to object using Research on threshold selection in picture,
With Boost method, feature extraction is carried out to the target image after background segment again, the target signature getting is passed through SVM method and built
Vertical familiar object template database is mated, thus obtaining the recognition result of familiar object；If matching result similarity is less than
Identification ratio, central control module 4 can send Cloud Server 5 pretreated image to by 4G communication module 3 to be known
Not, Cloud Server 5 carries out study identification by the database that the deep learning set up was trained, and is drawn according to highest similarity
Discrimination results.Local identification, distance measuring method can save SRT, and server mode may insure not familiar object
Accurately identify, the coordinated operation of two ways ensure that real-time, accuracy and popularity of image object identification etc..
Image capture module 1 is the binocular camera of simulation human eye, through USB diverter module 6 and central control module 4 phase
Even, obtain two width images of same scenery with the principle simulating mankind's right and left eyes visual imaging, the two width pictures getting be by
The jpeg form of binocular camera hardware-compressed, is then transferred to central control module 4, by two acquired in two cameras
Dimension image calculates the three-dimensional information of scenery with reference to neural network algorithm and extracts main information in picture, is obtained by BM algorithm
Go out image parallactic, it compares with the binocular vision of the mankind alike in principle.Because binocular camera shooting can provide observed object
Space multistory information, thus have a wide range of applications in field of machine vision.Meanwhile, the various signals that binocular camera shooting need to be carried out
Process the various aspects that relate to image processing techniques, comprehensive very strong.At present, conventional binocular machine vision system is substantially
It is using two optical axis are parallel or video camera of fixed angle to complete the collection of image.This mode has that structure is simple, surveys
The advantages of high precision that amount speed is fast, measures and mates.But system is before carrying out the measurements, need video camera is carried out strict
Parameter calibration.
The general principle of binocular range finding is as follows：
With reference to Fig. 3, setting the distance between human body and object is Z, the focal length f of camera, the camera lens of two cameras it
Between distance be T, object on the two sensors imaging xl and xr apart from d, be variable, have：
D=xl-xr
It is thus known that the value of d, you can Z is obtained according to similar triangle theory.
Binocular range finding is realized on Opencv, mainly comprises the following steps：
1. binocular correction and demarcation, obtain the parameter matrix of camera, demarcate and use MATLAB calibration tool；
2. Stereo matching, obtains disparity map；
3. adopt BM algorithm generate disparity map, if left and right match point is denser, match point is many, the image obtaining with former
Figure similarity is high, if match point ratio is sparse, the point obtaining is low with artwork similarity；
4. draw range finding；
5. the disparity map generating is input to reprojectImageTo3D () function, generates 3D point cloud, protect in 3D point cloud
There is a three-dimensional coordinate of 2D image, then reads the value of the z-axis in the three-dimensional coordinate of every two field picture, just drawn range data；
6. interpretation of result.
Target obstacle orientation discrimination：The Target Photo of background under Opencv environment, is obtained by threshold selection method,
And coordinate is set up to this picture whole region carry out region division, it is divided into three regions, respectively left front, front, right front；
Meanwhile, on the basis of the Target Photo removing background, target corresponding profile block diagram is generated according to target concrete shape, further according to
This block diagram, calculates the geometric center point of objective contour block diagram using geometrical principle；After the completion of central point and region division, according to
Objective contour central point to draw this target currently with respect to the concrete side of camera positioned at the concrete region of region division in figure
Position.
Binocular camera and satellite positioning module 7 are connected with central control module 4 through USB diverter module 6 respectively, and USB divides
Flow module 6 is FE1.1s, and FE1.1s is a highly integrated, high-quality, high-performance, low energy consumption, simultaneously or USB 2.0 4 ends at a high speed
The low cost solution of mouth line concentration.It adopts single exchange converter (Single Transaction Translator)
(STT) build to obtain more benefits.Six rather than two aperiodic translation caches are to reduce potential Communication Jamming.
Whole design is controlled based on state machine, to reduce the time delay of response；Not using microcontroller in this chip.For ensureing
High-quality, whole chip passes through to test scan chain (Test Scan Chain) even if under (480MHz) pattern at a high speed, making institute
Some logic elements are fully tested before shipment.Particularly being of Built In Self Test (Build-In-Self-Test) type order
Using all of high speed, at full speed, and low-speed mode AFE(analog front end) port (AFE) is also such in the packaging and testing stage.Low
The realization of energy consumption is by using 0.18m technology and integrated power supply/clock control mechanism.Most of pin does not need to count
When, unless be used；
The control block diagram of speech processing module 2 is as shown in figure 4, this module includes pronounciation processing chip (SYN6288).
Major function is exactly to realize the man-machine interaction of user and various functions intermodule, when user is apart from barrier
When distance is less than threshold value of warning, central control module 4 can send broadcast command to this module, and the data obtaining then is entered by this module
Row is processed, and is reported by outside speaker or ear speaker device, informs classification, the azran of user's preceding object thing
From for example " having vehicle at 2.3 meters of right front " so that user can carry out avoidance adjustment.When user wonders present bit
When putting, navigation button can be triggered, by the current geographical location information of voice broadcast.
Global position system is global positioning system (Global Positioning System) it is simply that using satellite pair
The technology that something is accurately positioned.Can ensure that at any time, on the earth, any point can observe 4 simultaneously and defend
Star, to realize the functions such as navigation, positioning, time service.The round-the-clock, high accuracy having and the feature of automatic measurement, as advanced
Measurement means and new productivity, incorporated the development of the national economy, national defense construction and social development each application neck
Domain.The operation principle of global position system is to measure the satellite of known location the distance between to receiver user, then comprehensive
The data closing multi-satellite knows the particular location of receiver.
GPS application in this technique, mainly makes glasses have accurate satellite positioning functions, its be achieved in that by
The client that glasses communicate as Socket, the latitude and longitude information that GPS is obtained utilizes 4G communication module 3 to pass through ICP/IP protocol
Pass to PC server end on internet, then the latitude and longitude information obtaining is passed through Baidu map by PC server again
API (Application Programming Interface) carries out mating of longitude and latitude and positional information, thus learning use
Person position, positional information is sent back client (central control module 4) by internet by last PC server end again.
As user it should be understood that during present position or wanting to allow household and friend will appreciate that oneself is currently located
During particular geographic location, trigger switch, 4G communication module 3 positional information will send close relative people's mobile phone in the way of note.Fig. 5
For satellite positioning module flow chart.
The realization of network communicating function, make use of the wireless broadband network access function of 4G communication module 3, the work(of this module
The foundation of communication link between glasses end and each function Cloud Server platform can be responsible for, one of them, be by glasses end and to be responsible for
The Cloud Server of identification range finding is set up and is in communication with each other in real time, and realizes voice call function, is made with facilitating its household to understand in time
User's current physical condition；Second, wireless communication module not only achieves the function of contacting with household, also achieve and GPS is obtained
The latitude and longitude information taking sent to PC server end by Internet and the reception server feedback information network communicating function,
And this function is also vital link in satellite fix link.
4G communication module 3 is connected with central control module 4 by USB interface, and USB main line bus signals are 4 modules and be
Communication interface between system.Driven according to the ECM mouth carrying of Linux3.2.0 kernel and ME3760_V2 module loaded,
The USB port using ME3760_V2 module is attached, and under linux, the ECM mouth of ME3760_V2 module is mapped to 5
Individual interface：ECM, AT, Modem, Log (are hereafter discussed in detail), wherein " " fall within the part of ECM mouth, for preventing
ECM function is capped, and should be filtered when carrying out USB initialization of (a) serial ports, and loads the PPP of linux kernel and drive so as to remaining
Interface initialization be USB serial equipment, finally use PPP instrument dial-up connection 4G network.
Technology realizes step：
1. the USB port of ME3760_V2 module is mapped to 5 interfaces under linux：ECM ,/, AT, Modem and Log；
2. kernel modifications, add the driving of ME3760_V2 module, such as on the basis of original linux version：Interpolation sets
Standby information, USB serial port drive filter ECM interface；
3. kernel compiling；
4. equipment loads, and the ECM mouth of ME3760_V2 module is mapped to network interface eth0, wherein AT/Modem/ in systems
Log mouth is mapped as ttyUSB0-ttyUSB2 in systems；
5. set up 4G to connect, using AT instruction, dialled using PPP, and the IP of static state setting eth0 and gateway DNS or make
With PPP dialing, just 4G network can be used.
6. central control module 4, this module mainly realizes the number that whole glasses various pieces functional unit is collected
Carry out Macro or mass analysis, and the scene being reflected according to data according to information, corresponding control instruction is sent to each functional module.
In order that modules operational efficiency is higher, performance is more stable, real-time performance is higher, team takes exclusively for core board
Carry LINUX PDA operating system, and its major function in the design has been embodied in following several aspects:
In the control of the IMAQ of binocular camera, use program setting one in advance in TQ210_COREB core board
Individual picture-taken frequency, and in real time by the compression of images having gathered and be sent to background server, core board constantly receives backstage
The image analysis result information that server passes over.
Go forward one by one in local-server the glasses end of identifying schemes, be equipped with simplifying the A8 of (SuSE) Linux OS first
On, using Opencv platform, familiar object is carried out with Boost method with feature extraction, and machine learning is used to the feature extracted
(SVM method) sets up matching template database.After identification distance measurement function is opened, the first picture decompression to pretreatment of glasses end,
Picture is split (Research on threshold selection) to object and background, then feature is carried out to the target image after background segment and carried
Take (Boost method), the target signature getting recycles SVM method to be mated with setting up familiar object template database,
Thus obtaining the recognition result of familiar object；If matching result similarity is less than identification ratio, glasses end can be pretreated
Image sends Cloud Server to by 4G network and is identified.Glasses end obstacle recognition, azimuth-range realize process as schemed
6.The distance of target, orientation obtain process ibid.
In the control to speech processing module 2, it is provided with threshold value of warning in background server analysis program in advance, when
When the object distance detecting is less than threshold value of warning, the text message receiving is obtained after voice module process by core board
Various functions sound bite, pass to user in the form of outside speaker or earphone, when detecting object distance is more than
During threshold value of warning, stop reporting.
When user is in and gets lost or be badly in need of the help obtaining friend relatives, pressed by touching the transmission of opsition dependent information
Key, after master control system obtains trigger, can adopt interrupt mode, control satellite positioning module 7 to determine the longitude and latitude of current location
Then latitude and longitude information is converted to specific geographical location information by central control module 4 by degree information again, leads to finally by 4G
Letter module 3 sends it to the household's phone number prestoring, and to facilitate household to understand its present position, carries out missing.
Cloud Server is by the constituted server cluster of the server of multiple stage parallel computation, has stronger operational capability.
The image analysis system of works passes through to build software platform on Cloud Server, using deep learning neutral net, sets up image
Learning database, the picture completing that transmission is come in is identified processing.Cloud Server, be one kind simply efficient, safe and reliable,
Disposal ability can elastic telescopic calculating service.Its way to manage is simpler than physical server efficient.User need not purchase in advance
Buy hardware, you can rapid establishment or any multiple stage Cloud Server of release.
Deep learning principle：Deep learning is in tranining database, it is desirable to provide powerful computing capability, and needs to deposit
Store up substantial amounts of training data, in view of Cloud Server has above advantage and characteristic, decide using Cloud Server as image recognition
The software platform of processing system.The concept of deep learning comes from the research of artificial neural network.Multilayer perceptron containing many hidden layers
It is exactly a kind of deep learning structure.Deep learning pass through to combine low-level feature formed more abstract high-rise represent attribute classification or
Feature, is represented with the distributed nature finding data.Deep learning is one of machine learning research new field, its motivation
It is to set up, simulation human brain is analyzed the neutral net that learns, it imitates the mechanism of human brain explaining data, such as image,
Sound and text.Fig. 6 is the deep learning model containing multiple hidden layers.
And in order to overcome the problem in neural metwork training, deep learning employs the training airplanes very different from neutral net
System.In traditional neural network, the mode using back propagation is carried out, and is exactly briefly the calculation using iteration
Method training whole network, random initialization, calculate the output of current network, then according between current output and layer
Difference goes to change the parameter of preceding layers, until restraining (being integrally a gradient descent method).And deep learning is one on the whole
The training mechanism of layer-wise.Reason for this is that, if using the mechanism of back propagation, for one
Individual deep neural network (more than 7 layers), the layer that residual error travels to foremost has become too little, so-called diffusion.
Fig. 7 is the training mechanism of layer-wise.
Deep learning training process is specific as follows：
(1) rise unsupervised learning (from the beginning of bottom, past top layer in layer is trained) using from lower：
Using no nominal data (have nominal data also can) each layer parameter of order training method, this step can be regarded as a nothing
Supervised training process, is that (this process can be regarded as the mistake of feature learning with traditional neural network difference the best part
Journey).Specifically, first with no nominal data training ground floor, (this layer can be regarded as first to learn the parameter of ground floor during training
Obtain a hidden layer making output and the minimum three-layer neural network of input difference), due to the restriction of model capacity and dilute
Thin property constrains so that the model obtaining can learn the structure to data itself, thus obtaining having more expression ability than input
Feature；After study obtains (n-1)th layer, using the output of n-1 layer as the input of n-th layer, train n-th layer, thus obtain respectively
Parameter to each layer；
(2) top-down supervised learning (goes to train by the data of tape label, the top-down transmission of error, to network
It is finely adjusted)：
Finely tune the parameter of whole multilayered model based on each layer parameter that the first step obtains further, this step is one prison
Superintend and direct training process；The first step is similar to the random initializtion initial value process of neutral net, because the first step of DL is not initial at random
Change, but obtained by the structure learning input data, thus this initial value is closer to global optimum such that it is able to obtain more
Good effect；So deep learning effect well largely gives the credit to the feature learning process of the first step.Training process adopts
CIFAR-10 natural scene storehouse.CIFAR-10 data set comprises the coloured image of 60000 32*32, has 10 classes.Have 50000
Individual training image and 10000 test images.Data set is divided into 5 training blocks and 1 test block, and each block has 10000 figures
Picture.Test block comprises 1000 images randomly choosing from every class.Training block comprises these images in a random order, but some
Training block may comprise more images than other classes.The training every class of block comprises 5000 images.
Outside CIFAR-10 natural scene storehouse, team adds common scene in some lives：Chair, dustbin,
People.So more can detect the practicality of identifying system.
The classification of non-familiar object, azimuth-range：
The non-common object type that Cloud Server platform is mainly responsible for front end None- identified is other, place's science and engineering of azimuth-range
Make.
Using Opencv platform, with Boost method, feature extraction is carried out to non-familiar object in backstage Cloud Server, and right
The feature extracted sets up matching template database with deep learning.After identification distance measurement function is opened, non-common object picture leads to
Cross 4G communication and upload to cloud server end, the picture decompression to pretreatment, in picture, (threshold is split to object and background
Value choosing method), then the target image after background segment is carried out with feature extraction (Boost method), the target signature getting
Again with set up non-familiar object template database and carry out deep learning, obtain similarity highest matching result, thus obtaining non-
The recognition result of familiar object；The distance of target, orientation obtain process ibid.
Claims (10)
1. a kind of smart guide glasses system is it is characterised in that include：
Image capture module (1), the ambient image around collection user, and the image of acquisition is uploaded to central control module
(4)；
Speech processing module (2), is connected with central control module (4), for the man-machine friendship of user and various functions intermodule
Mutually；
4G communication module (3), is glasses end and the Cloud Server (5) being responsible for identification range finding is set up and is in communication with each other in real time, and realizes
Voice call function；
Central control module (4), the data that each functional unit is collected and information carry out Macro or mass analysis, and according to data institute
The scene of reflection, sends corresponding control instruction to each functional module；Familiar object is mated by image template and parallax
Figure is identified and finds range；As distinguished, 4G communication module (3) is recycled to send to Cloud Server (5)；
Cloud Server (5), sets up image analysis system by building software platform on Cloud Server, using deep learning nerve
Network, sets up image study database, using deep learning and binocular location algorithm, obtain from the image uploading barrier away from
From with classification information, and feedback of the information is returned central control module (4)；
USB diverter module (6), for image capture module (1) is connected with central control module 4 with satellite positioning module (4),
Realize information exchange transmission.
2. according to claim 1 a kind of smart guide glasses system it is characterised in that：Also include satellite positioning module
(7), it is connected with central control module (4), the latitude and longitude information that GPS can also be obtained for satellite fix, 4G communication module (3)
Sent to PC server end and the reception server feedback information by Internet.
3. a kind of smart guide glasses blind-guiding method is it is characterised in that method is as follows：
Ambient image around user is gathered by image capture module (1), the image of acquisition is gone forward one by one using local-server
Identifying schemes：First by central control module (4), familiar object is mated by image template and disparity map is identified and finds range；
As distinguished, the network communicating function of 4G communication module (3) is recycled to send to Cloud Server (5), Cloud Server (5) utilizes
Deep learning and binocular range finding scheduling algorithm, obtain obstacle distance and classification information from the image uploading, and by information with literary composition
This form sends back central control module (4), then converts it into voice messaging by speech processing module (2), is accused by earphone
Know blind person.
4. according to claim 3 a kind of smart guide glasses blind-guiding method it is characterised in that：Described local-server is passed
Enter identifying schemes, on glasses end central control module (4), feature extraction is carried out to the picture of some common impairments things first
Carry out machine learning and set up matching template taxonomy database, after identification distance measurement function is opened, central control module (4) is first located to pre-
The picture decompression of reason, is split with background to object using Research on threshold selection in picture, then with Boost method to the back of the body
Target image after scape segmentation carries out feature extraction, and the target signature getting is passed through SVM method and set up familiar object template number
Mated according to storehouse, thus obtaining the recognition result of familiar object；If matching result similarity is less than identification ratio, central authorities control
Module (4) can send Cloud Server (5) pretreated image to by 4G communication module (3) and be identified, Cloud Server
(5) study identification is carried out by the database that the deep learning set up was trained, discrimination results are drawn according to highest similarity.
5. according to claim 3 a kind of smart guide glasses blind-guiding method it is characterised in that：
Image capture module (1) is the binocular camera of simulation human eye, through USB diverter module (6) and central control module (4) phase
Even, obtain two width images of same scenery with the principle simulating mankind's right and left eyes visual imaging, the two width pictures getting be by
The jpeg form of binocular camera hardware-compressed, is then transferred to central control module (4), by acquired in two cameras
Two dimensional image calculates the three-dimensional information of scenery with reference to neural network algorithm and extracts main information in picture, by BM algorithm
Draw image parallactic；
Binocular range finding is realized on Opencv, mainly comprises the following steps：
1. binocular correction and demarcation, obtain the parameter matrix of camera；
2. Stereo matching, obtains disparity map；
3. BM algorithm is adopted to generate disparity map；
4. draw range finding；
5. the disparity map generating is input to reprojectImageTo3D () function, generates 3D point cloud, preserve in 3D point cloud
The three-dimensional coordinate of 2D image, then read the value of the z-axis in the three-dimensional coordinate of every two field picture, just draw range data；
6. interpretation of result.
6. according to claim 3 a kind of smart guide glasses blind-guiding method it is characterised in that target obstacle orientation discrimination
Step is：Under Opencv environment, obtain the Target Photo of background by threshold selection method, and this picture whole region is built
Vertical coordinate carries out region division, is divided into three regions, respectively left front, front, right front；Meanwhile, in the mesh removing background
Target corresponding profile block diagram is generated according to target concrete shape, further according to this block diagram, using geometrical principle on mark picture basis
Calculate the geometric center point of objective contour block diagram；After the completion of central point and region division, it is located at according to objective contour central point
The concrete region of region division in figure is drawing this target currently with respect to the concrete orientation of camera.
7. according to claim 3 a kind of smart guide glasses blind-guiding method it is characterised in that：Speech processing module (2) is wrapped
Include pronounciation processing chip, major function is to realize the man-machine interaction of user and various functions intermodule, when user's distance barrier
When hindering the distance of thing to be less than threshold value of warning, central control module (4) can send broadcast command to this module, and this module then will obtain
Data processed, and reported by outside speaker or ear speaker device, informed classification, the side of user's preceding object thing
Position and distance.
8. according to claim 3 a kind of smart guide glasses blind-guiding method it is characterised in that：Satellite positioning module (7) with
The client that glasses communicate as Socket, the latitude and longitude information that GPS is obtained is assisted by TCP/IP using 4G communication module (3)
View passes to PC server end on internet, and then the latitude and longitude information obtaining is passed through Baidu map by PC server again
API carries out mating of longitude and latitude and positional information, thus learning user position, position is believed by last PC server end again
Breath sends back central control module (4) by internet；
The technology of network communicating function realizes step：
1. the USB port of ME3760_V2 module is mapped to 5 interfaces under linux：ECM ,/, AT, Modem and Log；
2. kernel modifications, add the driving of ME3760_V2 module on the basis of original linux version；
3. kernel compiling；
4. equipment loads, and the ECM mouth of ME3760_V2 module is mapped to network interface eth0, wherein AT/Modem/Log in systems
Mouth is mapped as ttyUSB0-ttyUSB2 in systems；
5. set up 4G to connect, using AT instruction, dialled using PPP, and the IP of static state setting eth0 and gateway DNS or use
PPP dials.
9. according to claim 3 a kind of smart guide glasses blind-guiding method it is characterised in that：To speech processing module
(2) in control, it is provided with threshold value of warning in background server analysis program in advance, when the object distance detecting is less than in advance
During alert threshold value, the sound bite of the various functions that the text message receiving is obtained after voice module process by core board,
Pass to user in the form of outside speaker or earphone, when detecting object distance is more than threshold value of warning, stop reporting.
10. according to claim 3 a kind of smart guide glasses blind-guiding method it is characterised in that：Deep learning principle is concrete
For, using Cloud Server as image recognition processing system software platform, deep learning pass through combine low-level feature formed more
Plus abstract high-rise expression attribute classification or feature, represented with the distributed nature finding data, deep learning is one on the whole
The training mechanism of individual layer-wise.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611039189.0A CN106389078A (en) | 2016-11-24 | 2016-11-24 | Intelligent blind guiding glass system and blind guiding method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611039189.0A CN106389078A (en) | 2016-11-24 | 2016-11-24 | Intelligent blind guiding glass system and blind guiding method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106389078A true CN106389078A (en) | 2017-02-15 |
Family
ID=58082552
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611039189.0A Pending CN106389078A (en) | 2016-11-24 | 2016-11-24 | Intelligent blind guiding glass system and blind guiding method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106389078A (en) |
Cited By (29)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106889710A (en) * | 2017-02-27 | 2017-06-27 | 东莞产权交易中心 | A kind of guide bracelet |
| CN107346629A (en) * | 2017-08-22 | 2017-11-14 | 贵州大学 | A kind of intelligent blind reading method and intelligent blind reader system |
| CN107392965A (en) * | 2017-07-24 | 2017-11-24 | 王华锋 | A kind of distance-finding method being combined based on deep learning and binocular stereo vision |
| CN108664266A (en) * | 2017-04-01 | 2018-10-16 | 深圳森若科技有限公司 | A kind of Portable manual intelligent apparatus and its configuration method |
| CN108761843A (en) * | 2018-05-29 | 2018-11-06 | 杭州视氪科技有限公司 | A kind of blind person's auxiliary eyeglasses detected for the water surface and puddle |
| CN108785029A (en) * | 2018-04-04 | 2018-11-13 | 厦门大学嘉庚学院 | A kind of intelligent interconnection formula guide vehicle |
| CN108960200A (en) * | 2018-07-31 | 2018-12-07 | 北京微播视界科技有限公司 | A kind of data processing method and electronic equipment based on intelligent interaction |
| CN109106563A (en) * | 2018-06-28 | 2019-01-01 | 清华大学天津高端装备研究院 | A kind of automation blind-guide device based on deep learning algorithm |
| CN109670445A (en) * | 2018-12-19 | 2019-04-23 | 宜视智能科技（苏州）有限公司 | Low vision aid intelligent glasses system |
| CN109662830A (en) * | 2019-01-18 | 2019-04-23 | 湖南师范大学 | A kind of language blind guiding stick, the deep neural network optimization method based on the walking stick |
| CN110045960A (en) * | 2018-01-16 | 2019-07-23 | 腾讯科技（深圳）有限公司 | Instruction set processing method, device and storage medium based on chip |
| CN110109457A (en) * | 2019-04-29 | 2019-08-09 | 北方民族大学 | A kind of intelligent sound blind-guidance robot control method and control system |
| CN110712187A (en) * | 2019-09-11 | 2020-01-21 | 珠海市众创芯慧科技有限公司 | Intelligent walking robot based on integration of multiple sensing technologies |
| CN110933308A (en) * | 2019-11-29 | 2020-03-27 | 维沃移动通信有限公司 | Photographing method, electronic device, and storage medium |
| CN111161507A (en) * | 2019-12-26 | 2020-05-15 | 潘文娴 | Danger sensing method and device based on intelligent wearable equipment and intelligent wearable equipment |
| CN111506067A (en) * | 2020-04-20 | 2020-08-07 | 上海电子信息职业技术学院 | Intelligent model car |
| CN111631927A (en) * | 2020-05-25 | 2020-09-08 | 浪潮金融信息技术有限公司 | Blind person aided navigation scheme based on visual information |
| CN112274400A (en) * | 2020-11-18 | 2021-01-29 | 郑州大学 | Intelligent blind guiding system |
| CN112370240A (en) * | 2020-12-01 | 2021-02-19 | 創啟社會科技有限公司 | Auxiliary intelligent glasses and system for vision impairment and control method thereof |
| CN112870033A (en) * | 2021-01-20 | 2021-06-01 | 江西理工大学南昌校区 | Intelligent blind guiding helmet system for unstructured road and navigation method |
| CN112932910A (en) * | 2021-01-25 | 2021-06-11 | 杭州易享优智能科技有限公司 | Wearable intelligent sensing blind guiding system |
| CN113032535A (en) * | 2019-12-24 | 2021-06-25 | 中国移动通信集团浙江有限公司 | Visual question and answer method and device for assisting visually impaired people, computing equipment and storage medium |
| CN113050917A (en) * | 2021-04-16 | 2021-06-29 | 中国科学技术大学 | Intelligent blind-aiding glasses system capable of sensing environment three-dimensionally |
| CN113208882A (en) * | 2021-03-16 | 2021-08-06 | 宁波职业技术学院 | Blind person intelligent obstacle avoidance method and system based on deep learning |
| CN113377097A (en) * | 2021-01-25 | 2021-09-10 | 杭州易享优智能科技有限公司 | Path planning and obstacle avoidance method for blind person guide |
| CN114191267A (en) * | 2021-12-06 | 2022-03-18 | 南通大学 | Light-weight intelligent method and system for assisting blind person in going out in complex environment |
| CN114587949A (en) * | 2022-02-21 | 2022-06-07 | 北京航空航天大学 | Blind guiding system |
| US11461587B2 (en) | 2021-01-13 | 2022-10-04 | International Business Machines Corporation | Intelligent visual recognition translation |
| CN115472039A (en) * | 2021-06-10 | 2022-12-13 | 上海博泰悦臻网络技术服务有限公司 | Information processing method and related product |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102592439A (en) * | 2012-02-03 | 2012-07-18 | 苏州大通汽车租赁有限公司 | Method and system for interactively calling taxi nearby by intelligent mobile equipment |
| CN102664819A (en) * | 2012-03-27 | 2012-09-12 | 陈奕龙 | Event publishing network system based on social mechanism and geographical position |
| CN103000024A (en) * | 2012-07-26 | 2013-03-27 | 苏州大通多宝软件技术有限公司 | Taxi reservation calling method and taxi reservation calling system |
| CN103976854A (en) * | 2014-05-15 | 2014-08-13 | 深圳市卡卓无线信息技术有限公司 | Intelligent blind glasses |
| JP2015164498A (en) * | 2014-02-28 | 2015-09-17 | 道芳 永島 | View aiding bone-conduction glasses for visually handicapped person |
| KR20150144510A (en) * | 2014-06-17 | 2015-12-28 | 김태형 | Glasses system for visually handicapped person |
| US20160030275A1 (en) * | 2014-07-31 | 2016-02-04 | Richplay Information Co., Ltd. | Blind-guiding positioning system for mobile device and method for operating the same |
| CN105342817A (en) * | 2015-12-17 | 2016-02-24 | 上海康龙宝康复器材科技有限公司 | Obstacle-avoidance glasses for blind people |
| CN105938257A (en) * | 2016-06-05 | 2016-09-14 | 苍南优医视眼镜有限公司 | GPS positioning instrument equipped glasses |
| CN106074099A (en) * | 2016-06-13 | 2016-11-09 | 李彤轩 | Blind person's special intelligent guides system |
-
2016
- 2016-11-24 CN CN201611039189.0A patent/CN106389078A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102592439A (en) * | 2012-02-03 | 2012-07-18 | 苏州大通汽车租赁有限公司 | Method and system for interactively calling taxi nearby by intelligent mobile equipment |
| CN102664819A (en) * | 2012-03-27 | 2012-09-12 | 陈奕龙 | Event publishing network system based on social mechanism and geographical position |
| CN103000024A (en) * | 2012-07-26 | 2013-03-27 | 苏州大通多宝软件技术有限公司 | Taxi reservation calling method and taxi reservation calling system |
| JP2015164498A (en) * | 2014-02-28 | 2015-09-17 | 道芳 永島 | View aiding bone-conduction glasses for visually handicapped person |
| CN103976854A (en) * | 2014-05-15 | 2014-08-13 | 深圳市卡卓无线信息技术有限公司 | Intelligent blind glasses |
| KR20150144510A (en) * | 2014-06-17 | 2015-12-28 | 김태형 | Glasses system for visually handicapped person |
| US20160030275A1 (en) * | 2014-07-31 | 2016-02-04 | Richplay Information Co., Ltd. | Blind-guiding positioning system for mobile device and method for operating the same |
| CN105342817A (en) * | 2015-12-17 | 2016-02-24 | 上海康龙宝康复器材科技有限公司 | Obstacle-avoidance glasses for blind people |
| CN105938257A (en) * | 2016-06-05 | 2016-09-14 | 苍南优医视眼镜有限公司 | GPS positioning instrument equipped glasses |
| CN106074099A (en) * | 2016-06-13 | 2016-11-09 | 李彤轩 | Blind person's special intelligent guides system |
Non-Patent Citations (3)
| Title |
|---|
| INSTHINK: "Phython-opencv之目标定位", 《CSDN》 * |
| JOE-QUAN: "完全基于Opencv的双目景深与测距的实现", 《CSDN》 * |
| 希仁娜.亚森 李湘: "基于Internet的远程数字图像处理", 《现代电子技术》 * |
Cited By (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106889710A (en) * | 2017-02-27 | 2017-06-27 | 东莞产权交易中心 | A kind of guide bracelet |
| CN108664266A (en) * | 2017-04-01 | 2018-10-16 | 深圳森若科技有限公司 | A kind of Portable manual intelligent apparatus and its configuration method |
| CN107392965A (en) * | 2017-07-24 | 2017-11-24 | 王华锋 | A kind of distance-finding method being combined based on deep learning and binocular stereo vision |
| CN107392965B (en) * | 2017-07-24 | 2020-10-30 | 王华锋 | Range finding method based on combination of deep learning and binocular stereo vision |
| CN107346629A (en) * | 2017-08-22 | 2017-11-14 | 贵州大学 | A kind of intelligent blind reading method and intelligent blind reader system |
| CN110045960A (en) * | 2018-01-16 | 2019-07-23 | 腾讯科技（深圳）有限公司 | Instruction set processing method, device and storage medium based on chip |
| CN108785029A (en) * | 2018-04-04 | 2018-11-13 | 厦门大学嘉庚学院 | A kind of intelligent interconnection formula guide vehicle |
| CN108761843A (en) * | 2018-05-29 | 2018-11-06 | 杭州视氪科技有限公司 | A kind of blind person's auxiliary eyeglasses detected for the water surface and puddle |
| CN109106563A (en) * | 2018-06-28 | 2019-01-01 | 清华大学天津高端装备研究院 | A kind of automation blind-guide device based on deep learning algorithm |
| CN108960200A (en) * | 2018-07-31 | 2018-12-07 | 北京微播视界科技有限公司 | A kind of data processing method and electronic equipment based on intelligent interaction |
| CN109670445A (en) * | 2018-12-19 | 2019-04-23 | 宜视智能科技（苏州）有限公司 | Low vision aid intelligent glasses system |
| CN109662830B (en) * | 2019-01-18 | 2019-11-29 | 湖南师范大学 | A kind of language blind guiding stick, the deep neural network optimization method based on the walking stick |
| CN109662830A (en) * | 2019-01-18 | 2019-04-23 | 湖南师范大学 | A kind of language blind guiding stick, the deep neural network optimization method based on the walking stick |
| CN110109457A (en) * | 2019-04-29 | 2019-08-09 | 北方民族大学 | A kind of intelligent sound blind-guidance robot control method and control system |
| CN110712187A (en) * | 2019-09-11 | 2020-01-21 | 珠海市众创芯慧科技有限公司 | Intelligent walking robot based on integration of multiple sensing technologies |
| CN110933308A (en) * | 2019-11-29 | 2020-03-27 | 维沃移动通信有限公司 | Photographing method, electronic device, and storage medium |
| CN113032535A (en) * | 2019-12-24 | 2021-06-25 | 中国移动通信集团浙江有限公司 | Visual question and answer method and device for assisting visually impaired people, computing equipment and storage medium |
| CN111161507A (en) * | 2019-12-26 | 2020-05-15 | 潘文娴 | Danger sensing method and device based on intelligent wearable equipment and intelligent wearable equipment |
| CN111506067A (en) * | 2020-04-20 | 2020-08-07 | 上海电子信息职业技术学院 | Intelligent model car |
| CN111631927A (en) * | 2020-05-25 | 2020-09-08 | 浪潮金融信息技术有限公司 | Blind person aided navigation scheme based on visual information |
| CN112274400A (en) * | 2020-11-18 | 2021-01-29 | 郑州大学 | Intelligent blind guiding system |
| CN112370240A (en) * | 2020-12-01 | 2021-02-19 | 創啟社會科技有限公司 | Auxiliary intelligent glasses and system for vision impairment and control method thereof |
| US11461587B2 (en) | 2021-01-13 | 2022-10-04 | International Business Machines Corporation | Intelligent visual recognition translation |
| CN112870033A (en) * | 2021-01-20 | 2021-06-01 | 江西理工大学南昌校区 | Intelligent blind guiding helmet system for unstructured road and navigation method |
| CN113377097A (en) * | 2021-01-25 | 2021-09-10 | 杭州易享优智能科技有限公司 | Path planning and obstacle avoidance method for blind person guide |
| CN112932910A (en) * | 2021-01-25 | 2021-06-11 | 杭州易享优智能科技有限公司 | Wearable intelligent sensing blind guiding system |
| CN113208882A (en) * | 2021-03-16 | 2021-08-06 | 宁波职业技术学院 | Blind person intelligent obstacle avoidance method and system based on deep learning |
| CN113050917A (en) * | 2021-04-16 | 2021-06-29 | 中国科学技术大学 | Intelligent blind-aiding glasses system capable of sensing environment three-dimensionally |
| CN113050917B (en) * | 2021-04-16 | 2022-10-28 | 中国科学技术大学 | Intelligent blind-aiding glasses system capable of sensing environment three-dimensionally |
| CN115472039A (en) * | 2021-06-10 | 2022-12-13 | 上海博泰悦臻网络技术服务有限公司 | Information processing method and related product |
| CN115472039B (en) * | 2021-06-10 | 2024-03-01 | 上海博泰悦臻网络技术服务有限公司 | Information processing method and related product |
| CN114191267A (en) * | 2021-12-06 | 2022-03-18 | 南通大学 | Light-weight intelligent method and system for assisting blind person in going out in complex environment |
| CN114587949A (en) * | 2022-02-21 | 2022-06-07 | 北京航空航天大学 | Blind guiding system |
| CN114587949B (en) * | 2022-02-21 | 2023-04-07 | 北京航空航天大学 | Blind guiding system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106389078A (en) | Intelligent blind guiding glass system and blind guiding method thereof | |
| CN106203390B (en) | A kind of intelligent blind auxiliary system | |
| CN107204010B (en) | A kind of monocular image depth estimation method and system | |
| CN105979203B (en) | A kind of multiple-camera cooperative monitoring method and device | |
| CN105912980B (en) | Unmanned plane and UAV system | |
| WO2022188379A1 (en) | Artificial intelligence system and method serving electric power robot | |
| CN107679522B (en) | Multi-stream LSTM-based action identification method | |
| CN110264416A (en) | Sparse point cloud segmentation method and device | |
| CN109034018A (en) | A kind of low latitude small drone method for barrier perception based on binocular vision | |
| CN107392965A (en) | A kind of distance-finding method being combined based on deep learning and binocular stereo vision | |
| CN111680588A (en) | Human face gate living body detection method based on visible light and infrared light | |
| CN107397658B (en) | Multi-scale full-convolution network and visual blind guiding method and device | |
| CN106022380A (en) | Individual identity identification method based on deep learning | |
| CN106214437A (en) | A kind of intelligent blind auxiliary eyeglasses | |
| CN110443898A (en) | A kind of AR intelligent terminal target identification system and method based on deep learning | |
| CN104700385B (en) | The binocular visual positioning device realized based on FPGA | |
| CN104287946A (en) | Device and method for prompting blind persons to avoid obstacles | |
| CN105686936A (en) | Sound coding interaction system based on RGB-IR camera | |
| CN105701811B (en) | A kind of acoustic coding exchange method based on RGB-IR cameras | |
| CN110135277B (en) | Human behavior recognition method based on convolutional neural network | |
| CN107578435A (en) | A kind of picture depth Forecasting Methodology and device | |
| CN114581817B (en) | Method and system for intelligently detecting wave height and wind speed from offshore wave monitoring video | |
| Gundewar et al. | A review on an obstacle detection in navigation of visually impaired | |
| CN107242964A (en) | Blind guiding system and method for work based on deep learning | |
| CN106971395A (en) | An a kind of three-dimensional sonar image modeling method poor based on a cloud average background |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170215 |
|
| RJ01 | Rejection of invention patent application after publication |