CN110070514B - FPGA (field programmable Gate array) embedded visual recognition method for blind road state and portable equipment - Google Patents
FPGA (field programmable Gate array) embedded visual recognition method for blind road state and portable equipment Download PDFInfo
- Publication number
- CN110070514B CN110070514B CN201910255813.8A CN201910255813A CN110070514B CN 110070514 B CN110070514 B CN 110070514B CN 201910255813 A CN201910255813 A CN 201910255813A CN 110070514 B CN110070514 B CN 110070514B
- Authority
- CN
- China
- Prior art keywords
- blind road
- blind
- image
- formula
- pixels
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000000007 visual effect Effects 0.000 title claims abstract description 13
- 238000001514 detection method Methods 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims abstract description 10
- 230000000877 morphologic effect Effects 0.000 claims description 20
- 239000011159 matrix material Substances 0.000 claims description 6
- 208000033749 Small cell carcinoma of the bladder Diseases 0.000 claims description 5
- 230000007797 corrosion Effects 0.000 claims description 5
- 238000005260 corrosion Methods 0.000 claims description 5
- 201000007710 urinary bladder small cell neuroendocrine carcinoma Diseases 0.000 claims description 5
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 18
- 230000000694 effects Effects 0.000 description 14
- 238000012545 processing Methods 0.000 description 8
- 238000005516 engineering process Methods 0.000 description 4
- 230000003628 erosive effect Effects 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 230000010339 dilation Effects 0.000 description 3
- 235000002566 Capsicum Nutrition 0.000 description 2
- 239000006002 Pepper Substances 0.000 description 2
- 241000722363 Piper Species 0.000 description 2
- 235000016761 Piper aduncum Nutrition 0.000 description 2
- 235000017804 Piper guineense Nutrition 0.000 description 2
- 235000008184 Piper nigrum Nutrition 0.000 description 2
- 239000004568 cement Substances 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 150000003839 salts Chemical class 0.000 description 2
- 238000010187 selection method Methods 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 208000010415 Low Vision Diseases 0.000 description 1
- 206010047571 Visual impairment Diseases 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000004438 eyesight Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000005469 granulation Methods 0.000 description 1
- 230000003179 granulation Effects 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000012535 impurity Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000004303 low vision Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000002245 particle Substances 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 208000029257 vision disease Diseases 0.000 description 1
- 230000004393 visual impairment Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
- A61H3/00—Appliances for aiding patients or disabled persons to walk about
- A61H3/06—Walking aids for blind persons
- A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
- G06T5/30—Erosion or dilatation, e.g. thinning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Physical Education & Sports Medicine (AREA)
- Epidemiology (AREA)
- Pain & Pain Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Rehabilitation Therapy (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Image Analysis (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses an FPGA embedded visual recognition method of a blind road state and portable equipment, wherein the blind road state comprises left and right deviation of a blind road relative to a user, right and left turning of the blind road and obstacles on the blind road, and the FPGA embedded visual recognition method comprises the following steps: the method comprises the steps of original video stream acquisition, HSV color space threshold segmentation, morphology open operation and judgment, blind road analysis, obstacle detection and voice prompt. The user informs the recognition result to the user through the portable equipment in a voice broadcasting mode through the method, and recognition of the blind road state is completed.
Description
Technical Field
The invention relates to a blind road intelligent identification technology, in particular to an FPGA (field programmable gate array) embedded visual identification method for a blind road state and portable equipment.
Background
The visually impaired population is a large, vulnerable group. The investigation shows that the number of the people with the visual disabilities in China reaches 1350 thousands, wherein more than 550 thousands of the people are blind people and account for 18 percent of the total number of the blind people in the world, and meanwhile, more than 800 thousands of the people with low vision exist. The number of the visual impairment crowds is huge, but the current situation of the national social blind guiding equipment still has a plurality of problems to be solved urgently. At present, the traditional blind guiding stick and the blind guiding dog are mainly used as the blind guiding assistive device occupying the market, however, the blind guiding stick is too limited in function, the controllability of the blind guiding dog is poor, the price is high, and novel blind guiding equipment is urgently needed by the society.
In domestic and foreign markets, on one hand, a few intelligent blind guiding products exist, most products have common problems of laggard sensing technology, less intellectualization and difficulty in man-machine interaction, and the improvement space is large. On the other hand, enterprises lack deep communication understanding with blind users, deviation exists in user experience and demand grasping, and a link of collecting user opinions and feeding back is weak.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an FPGA (field programmable gate array) embedded visual recognition method and portable equipment for a blind road state, which are used for solving the problem of technical defects of the blind guiding assistive device, in particular the problem of guiding the blind on the blind road.
The purpose of the invention is realized by adopting the following technical scheme:
an FPGA embedded visual recognition method for a blind road state comprises the following steps that the blind road deviates left and right relative to a user, the blind road is about to turn left and right, and obstacles exist on the blind road:
step 5, obstacle detection, namely performing connectivity detection on the bitmap and judging whether the blind road is shielded by obstacles or not;
and 6, voice prompt is carried out, the recognition result is informed to the user in a voice broadcasting mode, and the recognition of the blind road state is completed.
Preferably, in step 1, the original screen stream is 640 × 480 pixels and 30 frames/second RGB565 data, and the single frame image is 640 × 480 pixels RGB888 image.
Preferably, in step 2, the threshold segmentation applies the following discriminant:
in the formula, r, g and b respectively represent RGB color space parameters of an original image, and the value range of r, g and b is 0-255; h1 and H2 respectively represent the lower threshold and the upper limit of the H component in the HSV color space; s1 and S2 respectively represent the lower threshold and the upper limit of an S component in an HSV color space; v1 and V2 respectively represent the lower threshold and the upper limit of a V component in an HSV color space; max and min respectively represent the maximum value and the minimum value of the R, G and B values; the processed image is a bitmap of 640 x 480 pixels.
Preferably, in step 3, the morphological opening operation includes the following steps:
step 311, performing edge copy filling on the image, wherein each edge of the image is expanded by 2 pixels, and the filled image is a bitmap of 644 × 484 pixels;
step 312, performing morphological expansion on the image for 3 times, wherein a convolution kernel used for the expansion is a 5 × 5 full matrix;
313, performing morphological corrosion on the image for 2 times, wherein a convolution kernel used for the corrosion is a 5 multiplied by 5 full matrix;
step 314, discarding the filled edge pixels, and the processed image is a bitmap with 640 × 480 pixels.
Preferably, the step 3 of determining whether the vehicle is in a left-right deviation state or a left-right turning state includes the steps of:
step 321, comparing and judging the number of blind road components in the oblique window of the anchor point, finding out four vertexes of the blind road region by taking the horizontal and vertical coordinates of the anchor point as an auxiliary, and marking the upper right vertex as righttop, the upper left vertex as lefttop, the lower right vertex as rightdown and the lower left vertex as leftdown; recording an x-axis coordinate of the upper right vertex as righttop.x, recording a y-axis coordinate of the upper right vertex as righttop.y, recording an x-axis coordinate of the upper left vertex as lefttop.x, and recording a y-axis coordinate of the upper left vertex as lefttop.y;
step 322, determining whether the turning condition is satisfied by the following discriminant,
righttop.x-lefttop.x>＝2*(rightdown.x-leftdown.x) (2)
step 323, if formula 2 is satisfied, the specific turning direction can be judged through the following formula;
leftdown.x-lefttop.x>rightdown.x-leftdown.x (3)
righttop.x-rightdown.x>rightdown.x-leftdown.x (4)
if the formula 3 is met, the current blind road is in the left-turning direction; if the formula 4 is met, the current blind road is in the right turning direction;
in step 324, if the formula 2 is not satisfied, it can be determined whether the blind sidewalk is deviated from the center by the following formula,
(rightdown.x+leftdown.x)/2<leftThreshold.x (5)
(rightdown.x+leftdown.x)/2<rightThreshold.x (6)
wherein leftthreshold.x and rightthreshold.x are respectively threshold values of left and right side deviation; if the formula 5 is met, deviating the current blind road to the left side; if formula 6 is satisfied, the current blind road deviates to the right.
Preferably, in step 5, the connectivity detection includes dividing a rectangular frame in a bitmap of the blind road through four vertices of the blind road region, traversing a row of pixels twice in a forward and backward direction, and obtaining connectivity determination by combining the previous traversal accumulated values, and if the blind road does not meet the connectivity requirement, determining that an obstacle exists in the blind road.
Preferably, in step 6, the possible recognition result is pre-stored in the voice chip in advance, and the user is informed when the abnormal blind road state is recognized.
A portable device for implementing the identification method comprises an FPGA chip, a voice broadcast chip, a small CMOS camera and a body.
Compared with the prior art, the invention has the beneficial effects that: the method has higher reliability, and can effectively improve the accuracy of blind road state identification; the parallelism degree of the operation process is higher, and the recognition speed can be obviously improved by using the FPGA technology.
Drawings
FIG. 1 is a flow chart of an FPGA embedded visual recognition method for a blind road state;
FIG. 2 is an exemplary provided SCCB bus configuration timing simulation diagram;
FIG. 3 is an exemplary provided DVP bus timing simulation diagram;
FIG. 4 is an original video image captured in accordance with an embodiment of the present invention;
FIG. 5 is a schematic diagram of a line cache space according to the present invention;
FIG. 6 is a graph of a threshold segmentation effect provided by an embodiment of the present invention;
FIG. 7 is a schematic diagram of the morphological open operation of the present invention;
FIG. 8 is a diagram illustrating the effect of a morphological open operation according to an embodiment of the present invention;
FIG. 9 is a schematic diagram of a vertex selection method according to the present invention;
FIG. 10 is a diagram illustrating the effect of vertex selection according to an embodiment of the present invention;
fig. 11 is a diagram illustrating a detection effect of a connectivity obstacle according to an embodiment of the present invention;
FIG. 12 is a graph illustrating the effect of the treatment under the condition of strong light reflection according to the embodiment of the present invention;
fig. 13 is a diagram illustrating an effect of processing under a shadow condition according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Referring to fig. 1, an FPGA embedded visual recognition method for a blind road state includes that the blind road deviates left and right relative to a user, the blind road is about to turn left and right, and an obstacle exists on the blind road, and the FPGA embedded visual recognition method includes the following steps:
In step 1, the original screen stream is 640 × 480 pixels and 30 frames/second of RGB565 data, and the single frame image is a 640 × 480 pixels RGB888 image.
Referring to fig. 2 and 3, exemplary provided SCCB bus configuration timing simulation diagrams and DVP bus timing simulation diagrams, respectively.
Illustratively, referring to fig. 4 and 5, fig. 4 is an original video image captured by the embodiment of the present invention, and the image is stored in the line buffer space shown in fig. 5 inside the system, wherein the line buffer space is composed of a line number and image specific contents.
And 2, performing threshold segmentation on the image by an HSV color space threshold discriminant, wherein the processed image is a bitmap.
The blind sidewalk is an image with a remarkable color feature, so that information about the blind sidewalk can be acquired in the image by setting a color threshold. Because the RGB color model is designed from the angle of various colors for emitting and mixing, but is not close to the perception of human eyes for colors, and the threshold value is not easy to set, the HSV color model closer to the perception of human eyes is adopted, and the HSV component value is less influenced by the environment.
According to the meaning of each component of the HSV color space, H component is mainly considered in threshold segmentation, and S and V components are assisted to prevent areas with over-high or over-low brightness.
The conventional method is to convert the original RGB data into HSV space through conversion formulas (1 '), (2 ') and (3 '), and then to perform threshold judgment.
V＝max (3’)
Max and min in the formula represent the maximum value and the minimum value in the RGB three components, respectively.
However, the above method will generate 2 to 3 dividers, which will introduce a large combinational logic delay while causing a large overhead of system logic resources. Therefore, the above formula needs to be subjected to serial linear transformation, and the division operation which consumes a large amount of resources is changed into multiplication operation which can be completed by depending on internal DSP resources according to the characteristics of the FPGA.
The standard blind road should be vivid yellow with a hue approximately around 60 °, so that the H component threshold lower limit H1 falls within the interval (0 °,60 °), and the upper limit H2 falls within the interval (60 °,120 °). Since the S component is zero and the H component is meaningless, the lower limit S1 of the S component is a positive number.
After the above correction, in step 2, the discriminant of whether the pixel is located within the threshold range or the discriminant of the threshold segmentation application is as follows:
in the formula, r, g and b respectively represent RGB color space parameters of an original image, and the value range of the RGB color space parameters is 0-255; h1 and H2 respectively represent the lower threshold and the upper limit of the H component in the HSV color space; s1 and S2 respectively represent the lower threshold and the upper limit of an S component in an HSV color space; v1 and V2 respectively represent the lower threshold and the upper limit of a V component in an HSV color space; max and min respectively represent the maximum value and the minimum value of the R, G and B values; the processed image is a bitmap of 640 x 480 pixels.
Referring to fig. 6, a diagram of a threshold segmentation effect according to an embodiment of the present invention is provided. As can be seen, because the blind road is mostly made of cement and has fluctuation, a large amount of salt and pepper granulation and shadow exist. To eliminate the effect of the above problems, further processing of the image is required.
And 3, performing morphological opening operation and judgment, performing morphological opening operation on the bitmap, and judging whether blind road components exist or not.
In the morphological opening operation, in order to prevent loss of image edge information, edge copy filling should be performed on an image first. And because the blind road is mostly made of cement, the surface is rough and is easy to be attached with impurities, and a large amount of salt and pepper noise particles exist, a convolution kernel with a large range needs to be selected, and a rectangular window of 5 multiplied by 5 is selected in the text.
FIG. 7 is a schematic diagram of the morphological open operation of the present invention, including erosion and dilation operations using the method shown in the figure. After the adjacent five rows of data are selected, the convolution window slides across the whole data area from left to right, the convolution result consistent with the number of data columns is calculated, and the corrosion or expansion result can be obtained by setting a threshold value. After 3 dilations and 2 erosions, the image processing effect is best. Fig. 8 is a diagram illustrating an effect of a morphological opening operation according to an embodiment of the present invention.
The morphological filtering judgment method comprises the following steps: whether the angle difference is parallel or perpendicular is judged through the angle difference of the included angles of the straight line and the bottom edge, the angle difference is smaller than 10 degrees and is regarded as parallel, and the angle difference is within 85-95 degrees and is regarded as perpendicular. If more than 30% of the straight line relationships in a region are not parallel or perpendicular, then the region is not considered a blind road.
Specifically, in step 3, the morphological opening operation includes the following steps:
in step 311, the image is subjected to edge copy filling, each edge of the image is expanded by 2 pixels, and the filled image is a bitmap of 644 × 484 pixels.
In step 312, the image is morphologically dilated 3 times, and the convolution kernel used for dilation is a 5 × 5 full matrix.
Step 313, performing morphological erosion on the image for 2 times, wherein the convolution kernel used for the erosion is a 5 × 5 full matrix.
At step 314, the filled edge pixels are discarded, and the processed image is a bitmap of 640 × 480 pixels.
In step 3, the step of determining whether the vehicle is in a left-right deviation state or a left-right steering state includes the steps of:
step 321, comparing and judging the number of blind road components in the oblique window of the anchor point, finding out four vertexes of the blind road region by taking the horizontal and vertical coordinates of the anchor point as an auxiliary, and marking the upper right vertex as righttop, the upper left vertex as lefttop, the lower right vertex as rightdown and the lower left vertex as leftdown; the x-axis coordinate of the top right vertex is referred to as righttop.x, the y-axis coordinate of the top right vertex is referred to as righttop.y, the x-axis coordinate of the top left vertex is referred to as lefttop.x, and the y-axis coordinate of the top left vertex is referred to as lefttop.y.
Step 322, determining whether the turning condition is satisfied by the following discriminant,
righttop.x-lefttop.x>＝2*(rightdown.x-leftdown.x) (2)
Step 323, if formula 2 is satisfied, the specific turning direction can be judged through the following formula;
leftdown.x-lefttop.x>rightdown.x-leftdown.x (3)
righttop.x-rightdown.x>rightdown.x-leftdown.x (4)
if the formula 3 is met, the current blind road is in the left turning direction; if the formula 4 is satisfied, the current blind road is in the right turning direction.
In step 324, if the formula 2 is not satisfied, it can be determined whether the blind road deviates from the center through the following formula,
(rightdown.x+leftdown.x)/2<leftThreshold.x (5)
(rightdown.x+leftdown.x)/2<rightThreshold.x (6)
wherein leftthreshold.x and rightthreshold.x are respectively threshold values of left and right side deviation; if the formula 5 is met, deviating the current blind road to the left side; if formula 6 is satisfied, the current blind sidewalk deviates to the right.
The invention adopts the morphological opening operation and the morphological correlation judgment method to realize the noise reduction and the screening of the blind road area.
And 4, analyzing the blind road, judging the characteristics of the bitmap, and judging whether the blind road is in a left-right deviation state or a left-right steering state.
Fig. 9 is a schematic diagram of a vertex selection method of the present invention, which compares and determines the number of blind road components in an anchor point oblique window, assisted by the horizontal and vertical coordinates of the anchor point. Taking the top left vertex as an example, if the difference between the blind road component in the lower right window and the bottom left window is greater than a certain threshold value, and the sum of the horizontal and vertical coordinates of the point is minimum, the anchor point can be considered as the top left vertex of the blind road. FIG. 10 is a diagram illustrating the effect of vertex selection according to an embodiment of the present invention.
By selecting four vertexes of the blind road area, the left-right deviation and the left-right steering state of the blind road can be judged through formulas (5) and (6) and the following discriminant group.
righttop.x-lefttop.x>＝2*(rightdown.x-leftdown.x) (7)
leftdown.x-lefttop.x>rightdown.x-leftdown.x (8)
righttop.x-rightdown.x>rightdown.x-leftdown.x (9)
In the discriminant group, the upper right vertex is recorded as righttop, the upper left vertex is recorded as lefttop, the lower right vertex is recorded as rightdown, and the lower left vertex is recorded as leftdown; recording the x-axis coordinate of the top right vertex as righttop.x, recording the y-axis coordinate of the top right vertex as righttop.y, and so on; let ftthreshold. X, rightthreshold. X are the thresholds for left and right side deviation, respectively.
If the judgment result meets the judgment formula (7), the turning component exists in the current blind road; on the basis, if the judgment formula (8) is met, the current blind road is turned in the left direction; on the contrary, if the judgment expression (9) is satisfied, the current blind road is turned in the right direction.
If the judgment formula (7) is not satisfied, the current blind road is indicated to have no turning component; on the basis, if the judgment formula (5) is met, the current blind road deviates to the left side; otherwise, if the criterion (6) is satisfied, the current blind road deviates to the right.
And 5, detecting obstacles, detecting the connectivity of the alignment chart, and judging whether the blind road is shielded by the obstacles.
In step 5, the connectivity detection includes that a rectangular frame is marked off in a bitmap of the blind road through four vertexes of the blind road region, a certain row of pixels is traversed through forward and backward times, and connectivity judgment can be obtained by combining the previous traversal accumulated value, and if the blind road does not meet the connectivity requirement, the blind road is regarded as having an obstacle. See fig. 11 for a diagram of the effect of detecting a connectivity obstacle.
Specifically, the step 5 comprises the following steps:
step 511, dividing a maximum rectangular frame in the blind track bitmap through four vertexes of the blind track region, wherein the selection of the rectangular frame refers to the following formula:
beginrow＝max(lefttop.y,righttop.y) (10)
endrow＝min(leftdown.y,rightdown.y) (11)
begincol＝max(lefttop.x,leftdown.x) (12)
endcol＝min(rightdown.x,righttop.x) (13)
wherein beginrow, endrow, begincol and endcol respectively represent a start row, an end row, a start column and an end column.
Step 512, define an array and row variables to perform obstacle communication determination, where the length of the array is the same as the number of columns.
Step 513, if the pixel of a certain row is determined not to be the blind track content, the row variable is added and the relation between the row array variable value and the previous row array variable value is determined, if the previous row array variable value is greater than the row array variable value, the row array variable is updated to the previous row array variable value plus one, and if the previous row array variable value is less than or equal to the row array variable value, the row array variable is updated to the original value plus the row variable value. If the pixels in a certain row are determined as blind track contents, the row group variable and the column variable are zeroed.
Step 514, after traversing one row, reverse traversing is required, when the pixel of a certain row is determined not to be the blind track content and the variable value of the row array is smaller than the variable value of the array of the next row, the variable value of the row array is updated to the variable value of the array of the next row, and if the variable value of the array of the certain row exceeds a certain value (such as temporarily taking 300), it is determined that there is a barrier on the blind track ahead.
And 6, voice prompt is carried out, the recognition result is informed to the user in a voice broadcasting mode, and the recognition of the blind road state is completed.
In step 6, the possible recognition result is pre-stored in the voice chip in advance, and the user is informed when the blind road abnormal state is recognized.
Specifically, the system can comprehensively consider the judgment results of multiple processing by prerecording the possible processing results in the voice chip, and communicates the voice chip at a certain frequency when the abnormal state of the blind track is identified, so as to inform the user.
FIG. 12 is a graph illustrating the effect of the treatment under the condition of strong light reflection according to the embodiment of the present invention; fig. 13 is a diagram illustrating an effect of processing under a shadow condition according to an embodiment of the present invention.
A portable device for implementing the identification method comprises an FPGA chip, a voice broadcast chip, a small CMOS camera and a body.
The embodiment verification shows that the invention can meet the function requirement of a given system, and the overall processing speed is as high as more than 1000FPS per second. In addition, the system can adapt to complex road conditions, is relatively less restricted by the ambient light conditions, and has higher accuracy and robustness.
The invention is different from the traditional method that a blind guiding system utilizes a distance measuring sensor to judge, and provides a blind road state detection method based on embedded vision aiming at the conditions of left and right deviation, left and right steering and obstacle shielding of a blind road. Meanwhile, the FPGA technology is applied, and the parallel processing capacity and the recognition speed of the system are obviously improved.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (7)
1. A FPGA embedded visual recognition method for a blind road state is disclosed, wherein the blind road state comprises left and right deviation of the blind road relative to a user, right and left turning of the blind road and existence of obstacles on the blind road, and the FPGA embedded visual recognition method is characterized by comprising the following steps:
step 1, collecting an original video stream, configuring camera parameters through an SCCB bus, acquiring the original video stream data through a DVP bus and separating the original video stream data into continuous single-frame images;
step 2, performing threshold segmentation on the image through an HSV color space threshold discriminant, wherein the processed image is a bitmap;
step 3, performing morphology opening operation and judgment, performing morphology opening operation on the bitmap, and judging whether blind road components exist or not;
step 4, blind road analysis, feature judgment is carried out on the bitmap, and whether the blind road is in a left-right deviation state or a left-right steering state is judged;
step 5, obstacle detection, namely, carrying out connectivity detection on the alignment chart and judging whether the blind road is shielded by obstacles or not;
step 6, voice prompt is carried out, the recognition result is informed to a user in a voice broadcasting mode, and recognition of the blind road state is completed;
in step 3, the step of determining whether the vehicle is in a left-right deviation state or a left-right steering state includes the steps of:
step 321, comparing and judging the number of components of the blind road in the oblique window of the anchor point, taking the horizontal and vertical coordinates of the anchor point as an auxiliary, finding out four vertexes of the blind road area, and recording the upper right vertex as righttop, the upper left vertex as lefttop, the lower right vertex as rightdown and the lower left vertex as leftdown; recording an x-axis coordinate of the upper right vertex as righttop.x, recording a y-axis coordinate of the upper right vertex as righttop.y, recording an x-axis coordinate of the upper left vertex as lefttop.x, and recording a y-axis coordinate of the upper left vertex as lefttop.y;
step 322, determining whether the turning condition is satisfied by the following discriminant,
righttop.x - lefttop.x >＝ 2*(rightdown.x - leftdown.x) (2)
the formula (2) represents whether a transverse path exists on the basis of straight line of the blind road;
step 323, if the formula (2) is satisfied, the concrete turning direction can be judged through the following formula;
leftdown.x - lefttop.x > rightdown.x - leftdown.x (3)
righttop.x - rightdown.x > rightdown.x - leftdown.x (4)
if the formula (3) is met, the current blind road is in the left-turning direction; if the formula (4) is met, the current blind road is in the right turning direction;
in step 324, if the formula (2) is not satisfied, it can be determined whether the blind sidewalk is deviated from the center through the following formula,
(rightdown.x + leftdown.x) / 2 <leftThreshold.x (5)
(rightdown.x + leftdown.x) / 2 <rightThreshold.x (6)
wherein leftthreshold.x and rightthreshold.x are respectively the deviation threshold values of the left side and the right side; if the formula (5) is met, deviating the current blind road to the left side; if the formula (6) is satisfied, the current blind road deviates to the right.
2. The identification method according to claim 1, characterized in that: in step 1, the original video stream is 640 × 480 pixels, RGB565 data of 30 frames/second, and the single frame image is an RGB888 image of 640 × 480 pixels.
3. The identification method according to claim 1, characterized in that: in step 2, the following discriminant is applied to the threshold segmentation:
in the formula, r, g and b respectively represent RGB color space parameters of an original image, and the value range of r, g and b is 0-255; h1 and H2 respectively represent the lower limit and the upper limit of the H component in the HSV color space; s1 and S2 respectively represent the lower threshold and the upper limit of an S component in an HSV color space; v1 and V2 respectively represent the lower threshold and the upper limit of a V component in an HSV color space; max and min respectively represent the maximum value and the minimum value of the R, G and B values; the processed image is a bitmap of 640 x 480 pixels.
4. The identification method according to claim 1, characterized in that: in step 3, the morphological opening operation comprises the following steps:
step 311, performing edge copy filling on the image, wherein each edge of the image is expanded by 2 pixels, and the filled image is a bitmap of 644 × 484 pixels;
step 312, performing morphological expansion on the image for 3 times, wherein a convolution kernel used for the expansion is a 5 × 5 full matrix;
313, performing morphological corrosion on the image for 2 times, wherein a convolution kernel used for the corrosion is a 5 multiplied by 5 full matrix;
step 314, discarding the filled edge pixels, and the processed image is a bitmap with 640 × 480 pixels.
5. The identification method according to claim 1, characterized in that: in step 5, the connectivity detection includes that a rectangular frame is divided in a bitmap of the blind road through four vertexes of the blind road region, a certain row of pixels is traversed in positive and negative directions, the connectivity judgment can be obtained by combining the previous traversal accumulated value, and if the blind road does not meet the connectivity requirement, the blind road is regarded as having an obstacle.
6. The identification method according to claim 1, characterized in that: in step 6, the possible recognition result is pre-stored in the voice chip in advance, and the user is informed when the blind road abnormal state is recognized.
7. A portable device for implementing the identification method of any one of claims 1 to 6, characterized in that: the portable equipment comprises an FPGA chip, a voice broadcast chip, a small CMOS camera and a machine body.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910255813.8A CN110070514B (en) | 2019-04-01 | 2019-04-01 | FPGA (field programmable Gate array) embedded visual recognition method for blind road state and portable equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910255813.8A CN110070514B (en) | 2019-04-01 | 2019-04-01 | FPGA (field programmable Gate array) embedded visual recognition method for blind road state and portable equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110070514A CN110070514A (en) | 2019-07-30 |
| CN110070514B true CN110070514B (en) | 2023-04-04 |
Family
ID=67366793
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910255813.8A Active CN110070514B (en) | 2019-04-01 | 2019-04-01 | FPGA (field programmable Gate array) embedded visual recognition method for blind road state and portable equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110070514B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109938974A (en) * | 2019-04-01 | 2019-06-28 | 杨天歌 | A kind of visually impaired people is with portable intelligent guide equipment and blind-guiding method |
| CN110688915A (en) * | 2019-09-11 | 2020-01-14 | 上海卓希智能科技有限公司 | Safety device for independent traveling of visually impaired people |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150070129A1 (en) * | 2013-09-12 | 2015-03-12 | Toyota Motor Engineering & Manufacturing North America, Inc. | Methods and systems for providing navigation assistance to a user |
| CN105232295B (en) * | 2015-10-16 | 2017-03-22 | 北京机械设备研究所 | Intelligent guiding system for obstacle avoidance |
| CN106709518A (en) * | 2016-12-20 | 2017-05-24 | 西南大学 | Android platform-based blind way recognition system |
-
2019
- 2019-04-01 CN CN201910255813.8A patent/CN110070514B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110070514A (en) | 2019-07-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106709436B (en) | Track traffic panoramic monitoring-oriented cross-camera suspicious pedestrian target tracking system | |
| CN103400150B (en) | A kind of method and device that road edge identification is carried out based on mobile platform | |
| CN106682601B (en) | A kind of driver's violation call detection method based on multidimensional information Fusion Features | |
| WO2022027931A1 (en) | Video image-based foreground detection method for vehicle in motion | |
| US7747071B2 (en) | Detecting and correcting peteye | |
| WO2017084204A1 (en) | Method and system for tracking human body skeleton point in two-dimensional video stream | |
| CN109785291A (en) | A kind of lane line self-adapting detecting method | |
| CN110414385B (en) | Lane line detection method and system based on homography transformation and characteristic window | |
| CN109523480A (en) | A kind of defogging method, device, computer storage medium and the terminal of sea fog image | |
| JPH11288459A (en) | Method and device for detecting area such as face and observer tracking display | |
| CN1312625C (en) | Character extracting method from complecate background color image based on run-length adjacent map | |
| CN107424144A (en) | Weld joint tracking image processing algorithm based on laser vision | |
| CN110210451A (en) | A kind of zebra line detecting method | |
| CN107977645B (en) | Method and device for generating video news poster graph | |
| CN110070514B (en) | FPGA (field programmable Gate array) embedded visual recognition method for blind road state and portable equipment | |
| CN102842037A (en) | Method for removing vehicle shadow based on multi-feature fusion | |
| CN104318266A (en) | Image intelligent analysis processing early warning method | |
| CN105446333B (en) | Visual agv navigation system | |
| CN101650782A (en) | Method for extracting front human face outline based on complexion model and shape constraining | |
| CN104463138A (en) | Text positioning method and system based on visual structure attribute | |
| JP2000105819A (en) | Face image area detecting device | |
| CN105426816A (en) | Method and device of processing face images | |
| CN103218615A (en) | Face judgment method | |
| CN109948570B (en) | Real-time detection method for unmanned aerial vehicle in dynamic environment | |
| CN110472567A (en) | A kind of face identification method and system suitable under non-cooperation scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200414 Address after: Room 1119, 3255 Zhoujiazui Road, Yangpu District, Shanghai 200090 Applicant after: SHANGHAI JOOSEE SMART TECHNOLOGY Co.,Ltd. Address before: 201424 No. 999, Fengxian District, Shanghai Applicant before: Sun Xinyu |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |