CN111339956A - Height-limiting early warning method, device and system based on binocular camera and storage medium - Google Patents
Height-limiting early warning method, device and system based on binocular camera and storage medium Download PDFInfo
- Publication number
- CN111339956A CN111339956A CN202010126663.3A CN202010126663A CN111339956A CN 111339956 A CN111339956 A CN 111339956A CN 202010126663 A CN202010126663 A CN 202010126663A CN 111339956 A CN111339956 A CN 111339956A
- Authority
- CN
- China
- Prior art keywords
- early warning
- height
- interval
- limiting
- obstacle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G08—SIGNALLING
- G08G—TRAFFIC CONTROL SYSTEMS
- G08G1/00—Traffic control systems for road vehicles
- G08G1/16—Anti-collision systems
- G08G1/165—Anti-collision systems for passive traffic, e.g. including static obstacles, trees
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/204—Image signal generators using stereoscopic image cameras
- H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30261—Obstacle
Abstract
The invention discloses a binocular camera-based height-limiting early warning method, a device, a system and a storage medium, wherein the method comprises the following steps: acquiring an original image of a binocular camera within a preset range, and extracting a height-limiting barrier within the preset range through a height-limiting detection algorithm based on the original image; acquiring height information and distance information of the height-limiting barrier; judging the current early warning interval and interval crossing state of the height-limiting obstacle according to the distance information of the height-limiting obstacle and a preset sensing interval; and outputting a preset early warning instruction according to the current early warning interval and interval crossing state of the height-limiting barrier. The technical problem that a traditional binocular camera cannot give early warning in time during obstacle detection is solved.
Description
Technical Field
The invention relates to the technical field of binocular camera imaging, in particular to a height-limiting early warning method, device and system based on a binocular camera and a storage medium.
Background
With the development of sensor technology and machine vision technology, binocular cameras are increasingly widely applied in the fields of robots and intelligent automobiles. In recent years, accidents of casualties and property loss caused by collision between higher vehicle types such as buses, trucks, motor homes and the like and the height limiting rod are frequent. Because the shape, color, material and the like of the height limiting rod are not standardized and constrained by unified standards, the monocular camera is difficult to detect the height limitation, and the binocular stereo camera can accurately detect the height limitation barrier. However, the existing binocular stereo camera cannot effectively send out early warning prompts in time after detecting the obstacles, so that early warning information cannot be transmitted in time.
Disclosure of Invention
Therefore, the embodiment of the invention provides a height-limiting early warning method, device and system based on a binocular camera and a storage medium, so as to at least partially solve the technical problem that the traditional binocular camera cannot perform early warning in time during obstacle detection.
In order to achieve the above object, the embodiments of the present invention provide the following technical solutions:
a binocular camera-based height-limiting early warning method comprises the following steps:
acquiring an original image of a binocular camera within a preset range, and extracting a height-limiting barrier within the preset range through a height-limiting detection algorithm based on the original image;
acquiring height information and distance information of the height-limiting barrier;
judging the current early warning interval and interval crossing state of the height-limiting obstacle according to the distance information of the height-limiting obstacle and a preset sensing interval;
and outputting a preset early warning instruction according to the current early warning interval and interval crossing state of the height-limiting barrier.
Further, the acquiring of the original image of the binocular camera within the preset range and the extracting of the height-limiting barrier within the preset range by the height-limiting detection algorithm based on the original image specifically include:
acquiring an original image of a left eye camera and an original image of a right eye camera, and calculating the parallax in an image area of a superposed and effective part in the original image of the left eye camera and the original image of the right eye camera to obtain a parallax image of the original image of the binocular camera;
and extracting the height-limiting barrier by a height-limiting detection algorithm according to the original image of the left eye camera, the original image of the right eye camera and the phase parallax map of the original images.
Further, according to the distance information of the height-limiting obstacle and a preset sensing interval, the current early warning interval and interval crossing state of the height-limiting obstacle are judged, and the method specifically includes the following steps:
setting a preset sensing interval of the obstacle;
sequentially dividing a plurality of early warning intervals in the preset sensing interval, wherein the early warning intervals at least comprise a detection prompting interval, a primary early warning interval and an advanced early warning interval, and the detection prompting interval, the primary early warning interval and the advanced early warning interval are sequentially close to the height-limiting barrier in a straight-line distance;
and acquiring the current distance between a target vehicle and the height-limiting obstacle, and judging the early warning interval of the height-limiting obstacle relative to the target vehicle according to the current distance.
Further, according to the distance information of the height-limiting obstacle and a preset sensing interval, the current early warning interval and interval crossing state of the height-limiting obstacle are judged, and the method specifically includes the following steps:
acquiring position information of the height-limiting barrier in front and rear frames of an image, and judging the relative movement direction of a target vehicle;
and taking the target vehicle as a static coordinate system, judging that the section crossing occurs if the height-limiting barrier has any one of the following states, and otherwise, judging that the section crossing does not occur:
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a non-perception area;
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a detection prompt area;
and the height-limited barrier enters an advanced early warning interval from the primary early warning interval.
Further, the outputting a preset early warning instruction according to the early warning interval and the interval crossing state where the height-limiting obstacle is currently located specifically includes:
if the height-limiting barrier is judged to be in the detection prompt interval, an image early warning instruction is sent out so as to display an early warning icon with the height-limiting barrier in front through a screen;
if the height-limiting barrier is judged to be in the primary early warning interval, a primary early warning instruction and an image early warning instruction are sent out so as to trigger a primary early warning prompt tone and display an early warning icon with the height-limiting barrier in front through a screen;
and if the height-limited barrier is judged to be in the advanced early warning interval, sending a continuous early warning instruction and an image early warning instruction so as to continuously trigger a prompt early warning prompt tone and display an early warning icon of the height-limited barrier in front of the prompt tone through a screen.
Further, the method further comprises:
acquiring an early warning position of a target vehicle;
dividing the preset sensing interval into a plurality of early warning intervals according to the early warning positions;
the early warning intervals comprise a detection prompt interval, a primary early warning interval, a secondary early warning interval and an advanced early warning interval; the early warning positions comprise a first early warning position located between the detection prompt interval and the primary early warning interval and a second early warning position located between the primary early warning interval and the secondary early warning interval.
Further, the method for acquiring the early warning position of the target vehicle specifically comprises the following steps: calculating the first and second early warning positions by the following formulas:
wherein:
p1 is a primary early warning position, P2 is a secondary early warning position, and Pmax is a maximum detection position;
t is a time threshold;
v is the vehicle running speed, vup is the upper speed threshold and vlow is the lower speed threshold.
The invention also provides a height-limiting early warning device based on a binocular camera, which is used for implementing the method, and the device comprises:
the image acquisition unit is used for acquiring an original image of the binocular camera within a preset range and extracting a height-limiting barrier within the preset range through a height-limiting detection algorithm based on the original image;
the obstacle information acquisition unit is used for acquiring height information and distance information of the height-limiting obstacle;
the interval judgment unit is used for judging the current early warning interval and interval crossing state of the height-limiting obstacle according to the distance information of the height-limiting obstacle and a preset sensing interval;
and the early warning unit is used for outputting a preset early warning instruction according to the current early warning interval and interval crossing state of the height-limiting barrier.
The invention also provides a height-limiting early warning system based on the binocular camera, which comprises: a processor and a memory;
the memory is to store one or more program instructions;
the processor is configured to execute one or more program instructions to perform the method as described above.
The present invention also provides a computer storage medium having one or more program instructions embodied therein for executing the method as described above by a binocular camera based height limited early warning system.
According to the binocular camera-based height-limiting early warning method, device, system and storage medium, the binocular vision detection algorithm is used for obtaining the relevant information of the height-limiting barrier, and the early warning interval and the interval crossing state of the height-limiting barrier at present are judged based on the actual distance between the barrier and a target vehicle and the preset perception interval; therefore, a preset early warning instruction is output according to the current early warning interval and interval crossing state of the height-limiting barrier. The method comprises the steps of dividing an obstacle sensing range into a plurality of intervals, dividing the intervals into a plurality of grades, determining the interval according to the actual distance between an obstacle and a target vehicle so as to obtain the early warning interval where the obstacle is located, and triggering early warning when the obstacle crosses over or enters the advanced early warning interval in the interval. The early warning system can divide early warning information into a plurality of grades according to early warning intervals and interval crossing states, different early warning instructions are sent out according to different emergency grades, a driver is informed in time or a vehicle is braked and stopped, and the technical problem that a traditional binocular camera cannot give early warning in time when an obstacle is detected is solved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It should be apparent that the drawings in the following description are merely exemplary, and that other embodiments can be derived from the drawings provided by those of ordinary skill in the art without inventive effort.
The structures, ratios, sizes, and the like shown in the present specification are only used for matching with the contents disclosed in the specification, so as to be understood and read by those skilled in the art, and are not used to limit the conditions that the present invention can be implemented, so that the present invention has no technical significance, and any structural modifications, changes in the ratio relationship, or adjustments of the sizes, without affecting the effects and the achievable by the present invention, should still fall within the range that the technical contents disclosed in the present invention can cover.
Fig. 1 is a flowchart of a specific embodiment of a height-limiting early warning method provided in the present invention;
fig. 2 is a schematic structural diagram of early warning region division in the height-limiting early warning method provided by the present invention;
fig. 3 is a block diagram of a specific embodiment of a height-limiting early warning device provided in the present invention;
fig. 4 is a block diagram of an embodiment of a height-limited warning system according to the present invention.
Description of reference numerals:
100-image acquisition unit 200-obstacle information acquisition unit 300-section determination unit
400-early warning unit 500-early warning position calculation unit
Detailed Description
The present invention is described in terms of particular embodiments, other advantages and features of the invention will become apparent to those skilled in the art from the following disclosure, and it is to be understood that the described embodiments are merely exemplary of the invention and that it is not intended to limit the invention to the particular embodiments disclosed. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
According to the height-limiting early warning method based on the binocular camera, the early warning interval is divided into a plurality of early warning levels, different early warning strategies are corresponded, the early warning interval where the height-limiting barrier is located is obtained according to the actual distance between the height-limiting barrier and a target vehicle, the early warning emergency degree is judged according to the early warning interval where the height-limiting barrier is located and whether interval crossing conditions exist or not, and therefore a corresponding early warning instruction is output, and the technical problem that a traditional binocular camera cannot accurately early warn in time when the barrier is detected is solved.
In one embodiment, as shown in FIG. 1, the method comprises:
s1: the method comprises the steps of collecting an original image of a binocular camera within a preset range, and extracting height-limiting obstacles within the preset range through a height-limiting detection algorithm based on the original image. Acquiring original images of a binocular camera, namely acquiring original images of a left eye camera and an original image of a right eye camera to obtain two original images, calculating parallax in an image area of a superposed and effective part in the original images of the left eye camera and the right eye camera point by point to obtain a parallax image corresponding to the original images of the binocular camera; and extracting the height-limiting barrier by a height-limiting detection algorithm according to the original image of the left eye camera, the original image of the right eye camera and the phase disparity map of the original images. Firstly, projecting the disparity map on a plane where Y, Z is located along the X direction of a three-dimensional coordinate system to obtain a projected image; then, filtering out invalid projection points and invalid blobs on the projection images to obtain valid Blob images; and finally, back projecting the Blob image to a three-dimensional coordinate system, and determining the spatial position of the height limiting rod barrier so as to realize height measurement and calculation of the barrier.
S2: and acquiring the height information and the distance information of the height-limiting barrier, judging whether the target vehicle is ultrahigh according to the height information after acquiring the height information of the height-limiting barrier, and providing data support for the judgment of a subsequent early warning interval after acquiring the distance information of the height-limiting barrier.
S3: and judging the current early warning interval and interval crossing state of the height-limiting obstacle according to the distance information of the height-limiting obstacle and a preset sensing interval.
Wherein, in order to improve the interval division precision of early warning for interval division of early warning accords with actual service conditions's needs more, according to the distance information of limit for height barrier and predetermine the perception interval, judge the interval of early warning that limit for height barrier is located at present, specifically include following step:
s31: setting a preset sensing interval of the height-limiting barrier; in the running process of a target vehicle, the preset sensing interval is formed between the nearest detection position and the farthest detection position in front of the vehicle, and early warning is possibly triggered when the preset sensing interval enters the interval; the nearest detection position and the farthest detection position are set according to different equipment and user requirements, for example, the nearest detection position of a typical 12mm baseline and 8mm focal length binocular stereo camera is 3m, the farthest detection position is 80m, and a preset sensing interval of the height-limiting obstacle is formed between 80m and 3 m.
S32: in the preset perception interval, a plurality of early warning intervals are divided in sequence, the early warning intervals at least comprise a detection prompting interval, a primary early warning interval and an advanced early warning interval, and the detection prompting interval, the primary early warning interval and the advanced early warning interval are sequentially close to the height-limiting barrier in a linear distance, namely, the target vehicle is closer to the height-limiting barrier, and the alarm level of the early warning interval where the target vehicle is located is higher. In the actual use process, more intervals can be divided according to the required early warning precision, for example, a secondary early warning interval, a tertiary early warning interval and the like can be further divided between a primary early warning interval and a high-level early warning interval. The method comprises the following steps that a preset sensing interval is divided into a plurality of intervals for representing the levels of obstacles needing early warning, and the divided intervals are early warning intervals; as shown in fig. 2, a detection prompt interval is between the farthest detection position and the first-stage early warning position, a primary early warning interval is between the first-stage early warning position and the second-stage early warning position, and so on. In actual use, the early warning device can be divided into a detection prompt interval, a primary early warning interval and a highest early warning interval, a good early warning effect can be achieved, and the number of the specific early warning area divisions can be determined according to actual use requirements.
S33: acquiring the current distance between a target vehicle and the height-limiting obstacle, and judging the early warning interval of the height-limiting obstacle relative to the target vehicle according to the current distance; that is, after the current distance is obtained, comparing the distance range of which early warning interval the current distance is in, the height-limiting barrier is in which early warning interval; for example, the current distance is 30m, the secondary early warning interval is 20-40m according to the division of the early warning interval, and the height-limiting barrier is in the secondary early warning interval at the moment.
Further, in order to be able to timely know the interval crossing state of the height-limiting obstacle, so as to quickly trigger early warning when the interval crossing occurs, and ensure the timeliness of the early warning, the method specifically includes the following steps of judging the interval crossing state where the height-limiting obstacle is currently located according to the distance information of the height-limiting obstacle and a preset sensing interval:
s34: acquiring position information of the height-limiting barrier in front and rear frames of an image, and judging the relative movement direction of a target vehicle; the vehicle is taken as a static coordinate system, and the vehicle moves in front of the vehicle corresponding to an obstacle in the process of traveling. The aforementioned steps S1, S2, S31-S33 are performed for each frame of image acquisition, so as to determine whether the obstacle has performed a crossing operation according to the positions of the obstacles in the previous and subsequent frames.
S35: and taking the target vehicle as a static coordinate system, judging that the section crossing occurs if the height-limiting barrier has any one of the following states, and otherwise, judging that the section crossing does not occur:
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a non-perception area;
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a detection prompt area;
and the height-limited barrier enters an advanced early warning interval from the primary early warning interval.
The ride-through operation is defined as follows: firstly, a vehicle is taken as a static coordinate system, and an obstacle enters a primary/highest early warning interval from a detection prompting interval or a non-perception interval; secondly, the barrier moves to a higher-level interval in the sensing interval; the above condition is called interval crossing, and besides, the obstacle from the highest warning interval to the primary warning interval cannot be called interval crossing. In addition, when whether the early warning needs to be triggered or not is judged according to whether the interval crossing operation exists in the previous frame detection or not, when the obstacle is in the highest-level early warning interval, the early warning prompt tone can be triggered (prompted) whether the crossing operation exists or not.
S4: outputting a preset early warning instruction according to the current early warning interval and interval crossing state of the height-limiting barrier, outputting different types of early warning instructions when the height-limiting barrier is in different early warning intervals, and considering whether interval crossing exists; specifically, the method comprises the following steps:
if the height-limiting barrier is judged to be in the detection prompt interval, an image early warning instruction is sent out so as to display an early warning icon with the height-limiting barrier in front through a screen;
if the height-limiting barrier is judged to be in the primary early warning interval, a primary early warning instruction and an image early warning instruction are sent out so as to trigger a primary early warning prompt tone and display an early warning icon with the height-limiting barrier in front through a screen;
and if the height-limited barrier is judged to be in the advanced early warning interval, sending a continuous early warning instruction and an image early warning instruction so as to continuously trigger a prompt early warning prompt tone and display an early warning icon of the height-limited barrier in front of the prompt tone through a screen.
That is to say, when the height-limited barrier is in the detection prompting interval, the small screen of the device displays the early warning icon, and can prompt a driver that the height-limited barrier in front of the driver is avoided; the height-limiting barrier triggers a primary early warning prompt tone in the primary early warning interval, and the small screen of the equipment always displays an early warning icon; the height-limiting barrier can continuously trigger a sudden early warning prompt tone in the highest early warning interval, and the small screen of the equipment always displays early warning icons.
In the height-limiting early warning method provided by the invention, the early warning position is a key position for triggering early warning by an obstacle, and a buffer time is needed from the time of giving a prompt to a driver to the time of judging by the driver, so that the calculation and the acquisition of the early warning position are very important.
In order to realize the calculation and determination of the early warning position, thereby ensuring the accuracy and timeliness of the early warning, further, the method further comprises the following steps:
acquiring an early warning position of a target vehicle;
dividing the preset sensing interval into a plurality of early warning intervals according to the early warning positions;
the early warning intervals comprise a detection prompt interval, a primary early warning interval, a secondary early warning interval and an advanced early warning interval; the early warning positions comprise a first early warning position between the detection prompt interval and the primary early warning interval and a second early warning position between the primary early warning interval and the secondary early warning interval;
calculating the first and second early warning positions by the following formulas:
wherein:
p1 is a primary early warning position, P2 is a secondary early warning position, and Pmax is a maximum detection position;
t is a time threshold;
v is the vehicle running speed, vup is the upper speed threshold and vlow is the lower speed threshold.
In the above specific embodiment, the binocular camera-based height-limiting early warning method provided by the invention obtains the relevant information of the height-limiting obstacle by using a binocular vision detection algorithm, and judges the early warning interval and the interval crossing state of the height-limiting obstacle at present based on the actual distance between the obstacle and the target vehicle and the preset sensing interval; therefore, a preset early warning instruction is output according to the current early warning interval and interval crossing state of the height-limiting barrier. The method comprises the steps of dividing an obstacle sensing range into a plurality of intervals, dividing the intervals into a plurality of grades, determining the interval according to the actual distance between an obstacle and a target vehicle so as to obtain the early warning interval where the obstacle is located, and triggering early warning when the obstacle crosses over or enters the advanced early warning interval in the interval. The early warning system can divide early warning information into a plurality of grades according to early warning intervals and interval crossing states, different early warning instructions are sent out according to different emergency grades, a driver is informed in time or a vehicle is braked and stopped, and the technical problem that a traditional binocular camera cannot give early warning in time when an obstacle is detected is solved.
In addition to the above method, the present invention further provides a height-limiting early warning device based on a binocular camera, for implementing the above method, as shown in fig. 3, the device includes:
the image acquisition unit 100 is used for acquiring an original image of the binocular camera within a preset range, and extracting a height-limiting barrier within the preset range through a height-limiting detection algorithm based on the original image; acquiring original images of a binocular camera, namely acquiring original images of a left eye camera and an original image of a right eye camera to obtain two original images, calculating parallax in an image area of a superposed and effective part in the original images of the left eye camera and the right eye camera point by point to obtain a parallax image corresponding to the original images of the binocular camera; and extracting the height-limiting barrier by a height-limiting detection algorithm according to the original image of the left eye camera, the original image of the right eye camera and the phase disparity map of the original images.
An obstacle information acquiring unit 200 for acquiring height information and distance information of the height-limiting obstacle; after the height information of the height-limiting barrier is obtained, whether the target vehicle is ultrahigh can be judged according to the height information, and the distance information of the height-limiting barrier can provide data support for the judgment of the follow-up early warning interval.
And the interval determination unit 300 is configured to determine the early warning interval and the interval crossing state where the height-limiting obstacle is currently located according to the distance information of the height-limiting obstacle and the preset sensing interval.
Further, the section determination unit 300 is specifically configured to:
setting a preset sensing interval of the height-limiting barrier; in the running process of a target vehicle, the preset sensing interval is formed between the nearest detection position and the farthest detection position in front of the vehicle, and early warning is possibly triggered when the preset sensing interval enters the interval; the nearest detection position and the farthest detection position are set according to different equipment and user requirements, for example, the nearest detection position of a typical 12mm baseline and 8mm focal length binocular stereo camera is 3m, the farthest detection position is 80m, and a preset sensing interval of the height-limiting obstacle is formed between 80m and 3 m.
In the preset perception interval, a plurality of early warning intervals are divided in sequence, the early warning intervals at least comprise a detection prompting interval, a primary early warning interval and an advanced early warning interval, and the detection prompting interval, the primary early warning interval and the advanced early warning interval are sequentially close to the height-limiting barrier in a linear distance, namely, the target vehicle is closer to the height-limiting barrier, and the alarm level of the early warning interval where the target vehicle is located is higher. In the actual use process, more intervals can be divided according to the required early warning precision, for example, a secondary early warning interval, a tertiary early warning interval and the like can be further divided between a primary early warning interval and a high-level early warning interval. The method comprises the following steps that a preset sensing interval is divided into a plurality of intervals for representing the levels of obstacles needing early warning, and the divided intervals are early warning intervals; as shown in fig. 2, a detection prompt interval is between the farthest detection position and the first-stage early warning position, a primary early warning interval is between the first-stage early warning position and the second-stage early warning position, and so on. In actual use, the early warning device can be divided into a detection prompt interval, a primary early warning interval and a highest early warning interval, a good early warning effect can be achieved, and the number of the specific early warning area divisions can be determined according to actual use requirements.
Acquiring the current distance between a target vehicle and the height-limiting obstacle, and judging the early warning interval of the height-limiting obstacle relative to the target vehicle according to the current distance; that is, after the current distance is obtained, comparing the distance range of which early warning interval the current distance is in, the height-limiting barrier is in which early warning interval; for example, the current distance is 30m, the secondary early warning interval is 20-40m according to the division of the early warning interval, and the height-limiting barrier is in the secondary early warning interval at the moment.
Further, the section determination unit 300 is specifically configured to:
acquiring position information of the height-limiting barrier in front and rear frames of an image, and judging the relative movement direction of a target vehicle; the vehicle is taken as a static coordinate system, and the vehicle moves in front of the vehicle corresponding to an obstacle in the process of traveling.
And taking the target vehicle as a static coordinate system, judging that the section crossing occurs if the height-limiting barrier has any one of the following states, and otherwise, judging that the section crossing does not occur:
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a non-perception area;
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a detection prompt area;
and the height-limited barrier enters an advanced early warning interval from the primary early warning interval.
The ride-through operation is defined as follows: firstly, a vehicle is taken as a static coordinate system, and an obstacle enters a primary/highest early warning interval from a detection prompting interval or a non-perception interval; secondly, the barrier moves to a higher-level interval in the sensing interval; the above condition is called interval crossing, and besides, the obstacle from the highest warning interval to the primary warning interval cannot be called interval crossing. In addition, when whether the early warning needs to be triggered or not is judged according to whether the interval crossing operation exists in the previous frame detection or not, when the obstacle is in the highest-level early warning interval, the early warning prompt tone can be triggered (prompted) whether the crossing operation exists or not.
And the early warning unit 400 is configured to output a preset early warning instruction according to the early warning interval and the interval crossing state where the height-limiting obstacle is currently located. When the early warning interval is different, different types of early warning instructions are output, and whether interval crossing exists or not needs to be considered at the moment.
The early warning unit 400 is specifically configured to:
if the height-limiting barrier is judged to be in the detection prompt interval, an image early warning instruction is sent out so as to display an early warning icon with the height-limiting barrier in front through a screen;
if the height-limiting barrier is judged to be in the primary early warning interval, a primary early warning instruction and an image early warning instruction are sent out so as to trigger a primary early warning prompt tone and display an early warning icon with the height-limiting barrier in front through a screen;
and if the height-limited barrier is judged to be in the advanced early warning interval, sending a continuous early warning instruction and an image early warning instruction so as to continuously trigger a prompt early warning prompt tone and display an early warning icon of the height-limited barrier in front of the prompt tone through a screen.
That is to say, when the height-limited barrier is in the detection prompting interval, the small screen of the device displays the early warning icon, and can prompt a driver that the height-limited barrier in front of the driver is avoided; the height-limiting barrier triggers a primary early warning prompt tone in the primary early warning interval, and the small screen of the equipment always displays an early warning icon; the height-limiting barrier can continuously trigger a sudden early warning prompt tone in the highest early warning interval, and the small screen of the equipment always displays early warning icons.
In the height-limiting early warning device provided by the invention, the early warning position is a key position for triggering early warning by an obstacle, and a buffer time is needed from the time of giving a prompt to a driver to the time of judging by the driver, so that the calculation and the acquisition of the early warning position are very important.
In order to realize the calculation and determination of the early warning position, thereby ensuring the accuracy and timeliness of the early warning, further, the device further comprises an early warning position calculation unit 500, which is specifically configured to:
acquiring an early warning position of a target vehicle;
dividing the preset sensing interval into a plurality of early warning intervals according to the early warning positions;
the early warning intervals comprise a detection prompt interval, a primary early warning interval, a secondary early warning interval and an advanced early warning interval; the early warning positions comprise a first early warning position between the detection prompt interval and the primary early warning interval and a second early warning position between the primary early warning interval and the secondary early warning interval;
calculating the first and second early warning positions by the following formulas:
wherein:
p1 is a primary early warning position, P2 is a secondary early warning position, and Pmax is a maximum detection position;
t is a time threshold;
v is the vehicle running speed, vup is the upper speed threshold and vlow is the lower speed threshold.
In the above specific embodiment, the binocular camera-based height-limiting early warning device provided by the invention obtains the relevant information of the height-limiting obstacle by using a binocular vision detection algorithm, and judges the early warning interval and the interval crossing state of the height-limiting obstacle at present based on the actual distance between the obstacle and the target vehicle and the preset sensing interval; therefore, a preset early warning instruction is output according to the current early warning interval and interval crossing state of the height-limiting barrier. The method comprises the steps of dividing an obstacle sensing range into a plurality of intervals, dividing the intervals into a plurality of grades, determining the interval according to the actual distance between an obstacle and a target vehicle so as to obtain the early warning interval where the obstacle is located, and triggering early warning when the obstacle crosses over or enters the advanced early warning interval in the interval. The early warning system can divide early warning information into a plurality of grades according to early warning intervals and interval crossing states, different early warning instructions are sent out according to different emergency grades, a driver is informed in time or a vehicle is braked and stopped, and the technical problem that a traditional binocular camera cannot give early warning in time when an obstacle is detected is solved.
According to a third aspect of the embodiments of the present invention, the present invention further provides a height-limiting early warning system, as shown in fig. 4, the system includes: a processor 201 and a memory 202;
the memory is to store one or more program instructions;
the processor is configured to execute one or more program instructions to perform the method as described above.
In correspondence with the above embodiments, embodiments of the present invention also provide a computer storage medium containing one or more program instructions therein. Wherein the one or more program instructions are for execution by a height-limited early warning system to perform the method as described above.
In an embodiment of the invention, the processor may be an integrated circuit chip having signal processing capability. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete Gate or transistor logic device, discrete hardware component.
The various methods, steps and logic blocks disclosed in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The processor reads the information in the storage medium and completes the steps of the method in combination with the hardware.
The storage medium may be a memory, for example, which may be volatile memory or nonvolatile memory, or which may include both volatile and nonvolatile memory.
The nonvolatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory.
The volatile Memory may be a Random Access Memory (RAM) which serves as an external cache. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), SLDRAM (SLDRAM), and Direct Rambus RAM (DRRAM).
The storage media described in connection with the embodiments of the invention are intended to comprise, without being limited to, these and any other suitable types of memory.
Those skilled in the art will appreciate that the functionality described in the present invention may be implemented in a combination of hardware and software in one or more of the examples described above. When software is applied, the corresponding functionality may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer.
The above embodiments are only for illustrating the embodiments of the present invention and are not to be construed as limiting the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made on the basis of the embodiments of the present invention shall be included in the scope of the present invention.
Claims (10)
1. A binocular camera-based height limiting early warning method is characterized by comprising the following steps:
acquiring an original image of a binocular camera within a preset range, and extracting a height-limiting barrier within the preset range through a height-limiting detection algorithm based on the original image;
acquiring height information and distance information of the height-limiting barrier;
judging the current early warning interval and interval crossing state of the height-limiting obstacle according to the distance information of the height-limiting obstacle and a preset sensing interval;
and outputting a preset early warning instruction according to the current early warning interval and interval crossing state of the height-limiting barrier.
2. The height-limiting early warning method according to claim 1, wherein the acquiring of the original image of the binocular camera within a preset range and the extracting of the height-limiting obstacle within the preset range by a height-limiting detection algorithm based on the original image specifically comprises:
acquiring an original image of a left eye camera and an original image of a right eye camera, and calculating the parallax in an image area of a superposed and effective part in the original image of the left eye camera and the original image of the right eye camera to obtain a parallax image of the original image of the binocular camera;
and extracting the height-limiting barrier by a height-limiting detection algorithm according to the original image of the left eye camera, the original image of the right eye camera and the phase parallax map of the original images.
3. The height-limiting early warning method according to claim 2, wherein the judging of the early warning interval and the interval crossing state where the height-limiting obstacle is currently located according to the distance information of the height-limiting obstacle and a preset sensing interval specifically comprises:
setting a preset sensing interval of the obstacle;
sequentially dividing a plurality of early warning intervals in the preset sensing interval, wherein the early warning intervals at least comprise a detection prompting interval, a primary early warning interval and an advanced early warning interval, and the detection prompting interval, the primary early warning interval and the advanced early warning interval are sequentially close to the height-limiting barrier in a straight-line distance;
and acquiring the current distance between a target vehicle and the height-limiting obstacle, and judging the early warning interval of the height-limiting obstacle relative to the target vehicle according to the current distance.
4. The height-limiting early warning method according to claim 3, wherein the judging of the early warning interval and the interval crossing state where the height-limiting obstacle is currently located according to the distance information of the height-limiting obstacle and a preset sensing interval specifically comprises:
acquiring position information of the height-limiting barrier in front and rear frames of an image, and judging the relative movement direction of a target vehicle;
and taking the target vehicle as a static coordinate system, judging that the section crossing occurs if the height-limiting barrier has any one of the following states, and otherwise, judging that the section crossing does not occur:
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a non-perception area;
the height-limiting barrier enters a primary early warning area or a high-grade early warning area from a detection prompt area;
and the height-limited barrier enters an advanced early warning interval from the primary early warning interval.
5. The height-limiting early warning method according to claim 4, wherein the outputting of a preset early warning instruction according to the early warning interval and the interval crossing state where the height-limiting obstacle is currently located specifically comprises:
if the height-limiting barrier is judged to be in the detection prompt interval, an image early warning instruction is sent out so as to display an early warning icon with the height-limiting barrier in front through a screen;
if the height-limiting barrier is judged to be in the primary early warning interval, a primary early warning instruction and an image early warning instruction are sent out so as to trigger a primary early warning prompt tone and display an early warning icon with the height-limiting barrier in front through a screen;
and if the height-limited barrier is judged to be in the advanced early warning interval, sending a continuous early warning instruction and an image early warning instruction so as to continuously trigger a prompt early warning prompt tone and display an early warning icon of the height-limited barrier in front of the prompt tone through a screen.
6. The height-limiting early warning method according to any one of claims 1 to 5, further comprising:
acquiring an early warning position of a target vehicle;
dividing the preset sensing interval into a plurality of early warning intervals according to the early warning positions;
the early warning intervals comprise a detection prompt interval, a primary early warning interval, a secondary early warning interval and an advanced early warning interval; the early warning positions comprise a first early warning position located between the detection prompt interval and the primary early warning interval and a second early warning position located between the primary early warning interval and the secondary early warning interval.
7. The height-limiting early warning method according to claim 6, wherein the obtaining of the early warning position of the target vehicle specifically comprises: calculating the first and second early warning positions by the following formulas:
wherein:
p1 is a primary early warning position, P2 is a secondary early warning position, and Pmax is a maximum detection position;
t is a time threshold;
v is the vehicle running speed, vup is the upper speed threshold and vlow is the lower speed threshold.
8. A binocular camera based height-limiting early warning apparatus for implementing the method of any one of claims 1 to 7, the apparatus comprising:
the image acquisition unit is used for acquiring an original image of the binocular camera within a preset range and extracting a height-limiting barrier within the preset range through a height-limiting detection algorithm based on the original image;
the obstacle information acquisition unit is used for acquiring height information and distance information of the height-limiting obstacle;
the interval judgment unit is used for judging the current early warning interval and interval crossing state of the height-limiting obstacle according to the distance information of the height-limiting obstacle and a preset sensing interval;
and the early warning unit is used for outputting a preset early warning instruction according to the current early warning interval and interval crossing state of the height-limiting barrier.
9. The utility model provides a limit for height early warning system based on binocular camera which characterized in that, the system includes: a processor and a memory;
the memory is to store one or more program instructions;
the processor, configured to execute one or more program instructions to perform the method of any of claims 1-7.
10. A computer storage medium containing one or more program instructions for performing the method of any one of claims 1-7 by a binocular camera based height-limited early warning system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010126663.3A CN111339956A (en) | 2020-02-28 | 2020-02-28 | Height-limiting early warning method, device and system based on binocular camera and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010126663.3A CN111339956A (en) | 2020-02-28 | 2020-02-28 | Height-limiting early warning method, device and system based on binocular camera and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111339956A true CN111339956A (en) | 2020-06-26 |
Family
ID=71183956
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010126663.3A Pending CN111339956A (en) | 2020-02-28 | 2020-02-28 | Height-limiting early warning method, device and system based on binocular camera and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111339956A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113189993A (en) * | 2021-04-27 | 2021-07-30 | 北京石头世纪科技股份有限公司 | Self-walking equipment obstacle avoidance method, device, medium and electronic equipment |
| CN113370979A (en) * | 2021-07-09 | 2021-09-10 | 北京福田戴姆勒汽车有限公司 | Driving safety early warning method and system, vehicle, electronic equipment and storage medium |
| CN113592888A (en) * | 2021-07-01 | 2021-11-02 | 湖北文理学院 | Height limiting frame early warning method, device, equipment and storage medium |
| CN113781543A (en) * | 2021-11-10 | 2021-12-10 | 北京中科慧眼科技有限公司 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
| CN114463388A (en) * | 2022-04-13 | 2022-05-10 | 北京中科慧眼科技有限公司 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
| CN117590824A (en) * | 2024-01-19 | 2024-02-23 | 杭州市水务集团有限公司 | Sand setting tank control method and system in water purification process |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101986673A (en) * | 2010-09-03 | 2011-03-16 | 浙江大学 | Intelligent mobile phone blind-guiding device and blind-guiding method |
| CN106526604A (en) * | 2016-12-02 | 2017-03-22 | 机械科学研究总院青岛分院 | Safety protection system of walking mechanism |
| WO2018046015A1 (en) * | 2016-09-12 | 2018-03-15 | 中兴通讯股份有限公司 | Alarm method, device and terminal for vehicle |
| CN109238296A (en) * | 2018-08-16 | 2019-01-18 | 佛山科学技术学院 | A kind of high-precision indoor navigation method and device |
| CN109409311A (en) * | 2018-11-07 | 2019-03-01 | 上海为森车载传感技术有限公司 | A kind of limit for height method for early warning based on binocular stereo vision |
-
2020
- 2020-02-28 CN CN202010126663.3A patent/CN111339956A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101986673A (en) * | 2010-09-03 | 2011-03-16 | 浙江大学 | Intelligent mobile phone blind-guiding device and blind-guiding method |
| WO2018046015A1 (en) * | 2016-09-12 | 2018-03-15 | 中兴通讯股份有限公司 | Alarm method, device and terminal for vehicle |
| CN106526604A (en) * | 2016-12-02 | 2017-03-22 | 机械科学研究总院青岛分院 | Safety protection system of walking mechanism |
| CN109238296A (en) * | 2018-08-16 | 2019-01-18 | 佛山科学技术学院 | A kind of high-precision indoor navigation method and device |
| CN109409311A (en) * | 2018-11-07 | 2019-03-01 | 上海为森车载传感技术有限公司 | A kind of limit for height method for early warning based on binocular stereo vision |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113189993A (en) * | 2021-04-27 | 2021-07-30 | 北京石头世纪科技股份有限公司 | Self-walking equipment obstacle avoidance method, device, medium and electronic equipment |
| CN113189993B (en) * | 2021-04-27 | 2023-08-29 | 北京石头创新科技有限公司 | Obstacle avoidance method and device for self-walking equipment, medium and electronic equipment |
| CN113592888A (en) * | 2021-07-01 | 2021-11-02 | 湖北文理学院 | Height limiting frame early warning method, device, equipment and storage medium |
| CN113592888B (en) * | 2021-07-01 | 2024-02-02 | 湖北文理学院 | Height limiting frame early warning method, device, equipment and storage medium |
| CN113370979A (en) * | 2021-07-09 | 2021-09-10 | 北京福田戴姆勒汽车有限公司 | Driving safety early warning method and system, vehicle, electronic equipment and storage medium |
| CN113781543A (en) * | 2021-11-10 | 2021-12-10 | 北京中科慧眼科技有限公司 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
| CN114463388A (en) * | 2022-04-13 | 2022-05-10 | 北京中科慧眼科技有限公司 | Binocular camera-based height limiting device detection method and system and intelligent terminal |
| CN117590824A (en) * | 2024-01-19 | 2024-02-23 | 杭州市水务集团有限公司 | Sand setting tank control method and system in water purification process |
| CN117590824B (en) * | 2024-01-19 | 2024-04-02 | 杭州市水务集团有限公司 | Sand setting tank control method and system in water purification process |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111339956A (en) | Height-limiting early warning method, device and system based on binocular camera and storage medium | |
| US10043082B2 (en) | Image processing method for detecting objects using relative motion | |
| CN110069990B (en) | Height limiting rod detection method and device and automatic driving system | |
| CN112349144B (en) | Monocular vision-based vehicle collision early warning method and system | |
| EP2963615B1 (en) | Imaging device, and vehicle control device | |
| EP4089659A1 (en) | Map updating method, apparatus and device | |
| CN113792707A (en) | Terrain environment detection method and system based on binocular stereo camera and intelligent terminal | |
| CN109165629A (en) | It is multifocal away from visual barrier cognitive method, device, equipment and storage medium | |
| KR20080022748A (en) | Collision avoidance method using stereo camera | |
| JP2011048520A (en) | Device and method for monitoring vehicle periphery | |
| CN108021899A (en) | Vehicle intelligent front truck anti-collision early warning method based on binocular camera | |
| CN114495043B (en) | Method and system for detecting up-and-down slope road conditions based on binocular vision system and intelligent terminal | |
| CN110926408A (en) | Short-distance measuring method, device and system based on characteristic object and storage medium | |
| KR101749873B1 (en) | Method and apparatus for providing driving information using image of camera | |
| CN104115201A (en) | Three-dimensional object detection device | |
| CN112758067A (en) | Emergency braking method and system based on binocular stereo camera and intelligent terminal | |
| CN113140002B (en) | Road condition detection method and system based on binocular stereo camera and intelligent terminal | |
| US8983196B2 (en) | Vehicle periphery monitoring apparatus | |
| CN115661014A (en) | Point cloud data processing method and device, electronic equipment and storage medium | |
| CN113781543B (en) | Binocular camera-based height limiting device detection method and system and intelligent terminal | |
| JP4176558B2 (en) | Vehicle periphery display device | |
| JP3530803B2 (en) | Vehicle detection device | |
| CN113963066A (en) | Vehicle-mounted panoramic camera calibration method and device based on lane line and storage medium | |
| CN111881245B (en) | Method, device, equipment and storage medium for generating visibility dynamic map | |
| CN114463388A (en) | Binocular camera-based height limiting device detection method and system and intelligent terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |