CN116343176B - Pavement abnormality monitoring system and monitoring method thereof - Google Patents
Pavement abnormality monitoring system and monitoring method thereof Download PDFInfo
- Publication number
- CN116343176B CN116343176B CN202310617963.5A CN202310617963A CN116343176B CN 116343176 B CN116343176 B CN 116343176B CN 202310617963 A CN202310617963 A CN 202310617963A CN 116343176 B CN116343176 B CN 116343176B
- Authority
- CN
- China
- Prior art keywords
- abnormal
- area
- image
- identifying
- running
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005856 abnormality Effects 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000012544 monitoring process Methods 0.000 title claims abstract description 35
- 230000002159 abnormal effect Effects 0.000 claims abstract description 154
- 230000004044 response Effects 0.000 claims abstract description 46
- 238000001514 detection method Methods 0.000 abstract description 11
- 238000012545 processing Methods 0.000 abstract description 4
- 230000008859 change Effects 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 230000006399 behavior Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/008—Registering or indicating the working of vehicles communicating information to a remotely located station
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/085—Registering performance data using electronic data carriers
- G07C5/0866—Registering performance data using electronic data carriers the electronic data carrier being a digital video recorder in combination with video camera
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Traffic Control Systems (AREA)
Abstract
The application relates to a pavement anomaly monitoring system and a monitoring method thereof, belonging to the field of image processing. The method comprises the steps of obtaining a driving image collected by a driving recorder by a road surface abnormality monitoring method; identifying a road surface abnormal region in the running image based on the running image; determining a first area with the identification judgment probability higher than a first threshold value as a first type of pavement abnormal area; determining a running image when each second area appears for the first time as a target image, wherein the second area is an area with the identification judgment probability between a first threshold value and a second threshold value, and the second threshold value is lower than the first threshold value; acquiring continuous running images within a preset time after a target image is acquired; identifying abnormal response characteristics according to the continuous driving images; determining a second area in the target image before the continuous running image with the abnormal response characteristic as a second type of road surface abnormal area; and reporting the determined type and position of each road surface abnormal area under the condition of permission of a user. The accuracy of road surface abnormality detection can be improved.
Description
Technical Field
The application relates to the field of image processing, in particular to a pavement abnormality monitoring system and a monitoring method thereof.
Background
With the development of cities, roads become an important ring in urban construction and maintenance. In traditional city maintenance, the abnormality in the road is found through modes such as manual patrol, and the efficiency is low, and time and labor are wasted. With the development of the age, various informationized road surface abnormality detection means have been proposed. However, the related art pavement anomaly detection means often needs a large number of training models, and a strong calculation force or multiple sensing ways are used for supporting the pavement anomaly detection means, so that better accuracy can be ensured. On the other hand, some road surface anomalies that are not visually obvious are also difficult to detect and record, limited by the accuracy and cost of the sensor.
Disclosure of Invention
The application aims to provide a pavement anomaly monitoring system and a monitoring method thereof, which are used for solving the problem that the accuracy is not high enough in the traditional pavement anomaly detection.
The pavement anomaly monitoring method comprises the following steps:
acquiring a driving image acquired by a driving recorder;
identifying a road surface abnormal region in the driving image based on the driving image;
determining a first area with the identification judgment probability higher than a first threshold value as a first type of pavement abnormal area;
determining the running image when each second area appears for the first time as a target image, wherein the second area is an area with the identification judgment probability between the first threshold value and a second threshold value, and the second threshold value is lower than the first threshold value;
acquiring continuous running images within a preset time after the target image;
identifying abnormal response characteristics according to the continuous driving images;
determining that the second region in the target image before the continuous running image with the abnormal response characteristic is a second type of road surface abnormal region;
and reporting the determined type and position of each road surface abnormal area under the condition of permission of a user.
Further, the identifying of the abnormal response feature from the continuous running image includes:
and recognizing that the vehicle changes the driving path according to the continuous driving image.
Further, the identifying that the vehicle has changed the travel path based on the continuous travel image includes:
identifying a first travel path of the vehicle according to a travel image in front of the target image;
identifying a second travel path of the vehicle according to the continuous travel image after the target image;
comparing the first travel path and the second travel path;
and when the first running path and the second running path are different, determining that the running path of the vehicle is changed.
Further, the identifying the abnormal response feature according to the continuous running image includes:
identifying abnormal jitter characteristics according to the continuous driving images;
according to the abnormal response characteristics, determining that the second area in the target image is a second type of road surface abnormal area comprises:
and determining the second area as the second type pavement abnormal area according to the abnormal jitter characteristics.
Further, the determining, according to the abnormal shake feature, that the second area is the second type of road surface abnormal area includes:
identifying abnormal features of the second region according to the target image;
judging the association degree of the abnormal shaking characteristics and the abnormal characteristics of the second area, and determining the second area with the association degree higher than a third threshold value as the second type pavement abnormal area.
Further, the anomaly characteristic includes at least one of:
the size of the second region;
the degree of concavity and convexity of the second region.
Further, before the abnormal shake feature is identified according to the continuous running image, the method further includes:
identifying a second driving path of the vehicle according to the continuous driving image after the target image;
identifying whether the second area is on the second travel path;
and if the second area is positioned on the second running path, identifying abnormal shake features according to the continuous running images.
Further, the identifying of the abnormal response feature from the continuous running image includes:
and recognizing that the running speed of the vehicle is reduced by a fourth threshold value within a preset distance from the second area according to the continuous running images.
Further, the acquiring the continuous running image within the predetermined time after the target image includes:
calculating the driving-off time of the vehicle from the second area according to the driving speed of the vehicle;
and acquiring continuous running images with the duration not less than the driving-off time.
In another aspect, there is provided a pavement anomaly monitoring system comprising:
the image acquisition module is used for acquiring a driving image acquired by the driving recorder;
the first abnormality identification module is used for identifying a pavement abnormality area in the running image based on the running image; determining a first area with the identification judgment probability higher than a first threshold value as a first type of pavement abnormal area;
the second abnormality recognition module is used for determining the running image when each second area appears for the first time as a target image, wherein the second area is an area with recognition judgment probability between the first threshold value and a second threshold value, and the second threshold value is lower than the first threshold value; acquiring continuous running images within a preset time after the target image; identifying abnormal response characteristics according to the continuous driving images; determining that the second region in the target image before the continuous running image with the abnormal response characteristic is a second type of road surface abnormal region;
and the abnormality reporting module is used for reporting the determined type and position of each pavement abnormality area under the condition of permission of a user.
According to the road surface abnormality monitoring method provided by the application, the road surface abnormality area is identified by acquiring the running image acquired by the automobile data recorder, the second area in which the identification judgment probability is between the first threshold value and the second threshold value is further identified based on the continuous running image, after the abnormal response characteristic is identified according to the continuous running image, the second area in one target image before the continuous running image with the abnormal response characteristic is determined to be the second type road surface abnormality area, high-precision sensing equipment is not needed, and the accuracy of road surface abnormality detection can be improved.
Drawings
Fig. 1 is a flow chart of a pavement anomaly monitoring method according to an embodiment of the present application;
FIG. 2 is a detailed flow chart of part of steps of a pavement anomaly monitoring method according to an embodiment of the present application;
FIG. 3 is a schematic diagram showing a comparison of a vehicle driving path according to an embodiment of the present application;
FIG. 4 is a detailed flowchart of another part of the steps of the method for monitoring abnormal road surface according to the embodiment of the present application;
fig. 5 is a schematic structural diagram of a pavement anomaly monitoring system according to an embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Embodiment one:
the present embodiment provides a method for monitoring road surface abnormality, referring to fig. 1, the method includes:
s11, acquiring a driving image acquired by a driving recorder;
the driving image may be acquired by a processing unit of the driving recorder itself, or may be acquired by an operating system of the vehicle, that is, the method for monitoring the road surface abnormality in this embodiment may be executed by an operating system including, but not limited to, the driving recorder or the vehicle.
It will be appreciated that, as long as there is a running image, the content of the image can be analyzed to obtain an abnormal condition of the road surface, so in this embodiment, the analysis of the running image may be real-time, or any other time before the running image is deleted, which is not limited in this embodiment.
S12, identifying a pavement abnormal area in the running image based on the running image;
in this embodiment, the identification of the abnormal area of the road surface may be performed in any manner, but it should be noted that, in the algorithm for identifying the abnormal area of the road surface adopted in this embodiment, the identification is performed based on probability decision. That is, the embodiment performs probability judgment on each possible area of the road surface, wherein the area with higher judgment probability is more likely to have a real abnormal state, and the area with low judgment probability is not in an abnormal state. Of course, in the conventional method for identifying the abnormal region of the pavement based on the image, the identification result is generally shown directly according to the decision threshold, and the region higher than the decision threshold is identified as the abnormal region, so that the decision probability of each region as the abnormal region is not marked directly.
S13, determining a first area with the identification judgment probability higher than a first threshold value as a first type of pavement abnormal area;
the first threshold is set according to the situation, for example, the probability that the first threshold corresponds to is not less than 90%, for example, 90%, 95%, etc., and the accuracy of taking the first threshold as the decision threshold is about 90% or 95%. Of course, in practical application, the first threshold is not limited to the probability percentage representation, but may be any other form of decision threshold. In this embodiment, if the recognition judgment probability is higher than the first threshold value, the first region is an abnormal region, and in the present application, these directly detected abnormal regions are determined as the first type road surface abnormal regions.
S14, determining a running image when each second area appears for the first time as a target image;
the second region is a region with the identification judgment probability between a first threshold value and a second threshold value, and the second threshold value is lower than the first threshold value. It can be understood that, under the same identification algorithm, the second region referred to in this embodiment has a lower possibility of abnormality than the first region. In practical applications, the second threshold is also set according to requirements, for example, the probability that the second threshold corresponds to is not less than 75%, for example, 75%, 80%, etc. In the existing anomaly detection, a first region with an identification decision probability above a first threshold is identified as an anomalous region, and the remaining regions are identified as no anomaly. However, in this embodiment, the running image of the second area between the first threshold and the second threshold is determined as the target image, and further analysis is performed accordingly, so that some areas suspected of being abnormal can be further confirmed, and the accuracy of detection is improved.
S15, acquiring continuous running images within a preset time after the target image is acquired;
s16, identifying abnormal response characteristics according to the continuous driving images;
it can be understood that if the vehicle is running normally on the road, the images collected by the automobile data recorder show a certain regularity, for example, the change of each frame of image is uniform and smooth, and each reference object is regularly displaced frame by frame. When the vehicle is disturbed or operated from the outside, the images collected by the automobile data recorder show abnormal regularity, for example, the vehicle can shake when running on a damaged road, and the generated shake can be reflected on the running data recorder, so that the smooth and uniform change of each frame of images is destroyed; also, for example, the vehicle suddenly decelerates or turns, and the speed at which the position of each reference object in the image shifts or displaces suddenly changes. These image change characteristics different from those during normal running, that is, the abnormal response characteristics, are set according to the actual conditions, and in practical applications, the degree of shake, vehicle steering, and speed change are abnormal.
S17, determining a second area in one target image before the continuous running image with the abnormal response characteristic is a second type road surface abnormal area;
for example, after the running image identifies the second area, continuing to acquire the continuous running image after the second area appears, and if the abnormal response feature is identified in the continuous running image immediately after the second area appears, determining the second area as a second type of road surface abnormal area.
It will be appreciated that the continuous travel image in which the abnormal response feature appears is associated with a certain or a certain set of second regions, and that only the second region corresponding to the continuous travel image in which the abnormal response feature appears is determined as a second type of road surface abnormal region. As for the second area earlier, there is no relation with the continuous running image in practice, so in the present embodiment, only the second area in the previous one of the target images needs to be determined as the second-type road surface abnormality area.
It is understood that if no abnormal response feature occurs in the continuous running image, the corresponding second region is considered to be free from road surface abnormality.
S18, reporting the determined types and positions of the abnormal areas of the pavement under the condition of permission of a user;
it is understood that the location of the vehicle may relate to user privacy, and thus, whether to allow reporting of location information or not may be confirmed to the user in advance, or an operation of reporting of a road surface abnormality may be actively performed by the user. In some embodiments, if the user allows, the driving image of the driving recorder may also be reported. The road surface abnormal region in the application comprises but is not limited to a first road surface abnormal region and a second road surface abnormal region which are confirmed, wherein the types of the road surface abnormal region are respectively a first type and a second type.
For example, for a vehicle event data recorder with positioning and networking functions, the vehicle event data recorder can acquire the position of the abnormal road surface area and report the position. In some embodiments, the operation system of the vehicle may also acquire the image of the automobile data recorder and perform the operations of positioning and reporting.
In some implementations, the analysis of the driving image is not performed in real time, and the driving image of the vehicle recorder is stored in association with the position information of the driving image when the driving image is captured. When the road surface abnormality is analyzed from the running image in the subsequent analysis, the position information corresponding to the section of running image is acquired, so that the road surface abnormality can be identified and reported in a non-real-time manner.
Optionally, identifying the abnormal response feature from the continuous running image includes:
it is recognized that the vehicle has changed the travel path from the continuous travel image.
The continuous running images in this embodiment each refer to a continuous running image after the target image is determined, which can reflect the running condition of the vehicle within a certain time after the second region appears. It should be noted that the vehicle changes its travel path including, but not limited to, changing lanes, or turning the vehicle substantially or changing lanes and returning to the original lane of travel. As one embodiment, whether the travel path of the vehicle is changed may be identified by a reference in the continuous travel image, and by way of example, a road line is identified from the continuous travel image, and whether the travel direction of the vehicle is deviated may be identified by taking the road line as a reference, thereby determining whether the travel path is changed. As an example, when the relative position of the road line to the vehicle is shifted by a predetermined distance in the continuous running image, or the relative angle thereof to the vehicle is changed by a predetermined angle, the vehicle may be considered to change the running path.
Alternatively, as shown in fig. 2, identifying the vehicle to change the travel path from the continuous travel image includes:
s21, recognizing a first travel path of the vehicle according to a travel image in front of the target image;
s22, identifying a second driving path of the vehicle according to the continuous driving images after the target image;
s23, comparing the first travel path with the second travel path;
and S24, when the first running path and the second running path are different, determining that the running path of the vehicle is changed.
As an example, as shown in fig. 3, a first travel path a of the vehicle is identified from a travel image preceding the target image. It will be appreciated that the first travel path a is a predicted path based on the relative position and orientation of the vehicle with respect to the road line or other reference in the travel image preceding the target image. For example, if the vehicle is traveling in the first lane R1 while the traveling image in front of the target image is always kept, the first traveling path a is predicted to be a path along the first lane R1 that normally travels. In this example, the second driving path B identified by the continuous driving image after the target image is displayed, the vehicle changes the lane to the second lane R2, and then changes the lane back to the first lane R1, and obviously, the vehicle has a behavior of deviating from the original normal driving path during the driving process, and the driving path is changed. After the vehicle is identified to change the driving path, the second area is determined to be a second type of abnormal area of the road surface.
Optionally, identifying the abnormal response feature according to the continuous running image includes:
identifying abnormal shake features according to the continuous running images;
according to the abnormal response characteristics, determining that the second area in the target image is the second type of road surface abnormal area comprises:
and determining the second area as a second type of road surface abnormal area according to the abnormal shake characteristics.
Due to the fact that the stability of the fixation of the automobile data recorder is different, and the automobile has a certain shake in the normal running process, especially for vehicles with long service time, the automobile frame can be loose. To ensure accurate recognition of abnormal vehicle shake, in some examples, changes in the running image during normal running of the vehicle are learned in advance, and shake features during normal running of the vehicle are extracted. If the image change condition in the continuous running image does not coincide with the image feature of the normal running of the vehicle, the portion of the difference may belong to abnormal shake, and, for example, may be determined as an abnormal shake feature when the abnormal shake is greater than a certain amplitude or a preset condition is satisfied. Through the pre-learning of the characteristics of the normal running process of the vehicle, the shake caused by the unstable installation of the vehicle or the automobile data recorder is eliminated, and the relevance of the identified abnormal shake characteristics and the road surface abnormality can be improved, so that the accuracy of road surface abnormality detection is improved.
Optionally, determining the second area as a second type of abnormal area according to the abnormal shake feature includes:
identifying abnormal features of the second region according to the target image;
judging the association degree of the abnormal shake features and the abnormal features of the second region, and determining the second region with the association degree higher than a third threshold value as a second type road surface abnormal region.
It should be noted that the higher the association of the abnormal shake feature with the abnormal feature of the second region, the more likely the shake is due to the vehicle passing through the second region. For example, the association degree may be expressed as a parameter value according to a predetermined rule, and the specific association degree calculation rule or formula is not limited in the present application, and may be set according to actual situations or trained by deep learning or the like.
Optionally, the anomaly characteristic comprises at least one of:
the size of the second region;
degree of concavity and convexity of the second region.
For example, if the second area is recognized as a small area according to the target image, and then the driving image shows that the vehicle generates abnormal shake for a long time, but the vehicle has already driven away from the position of the second area, the association degree between the two areas is judged to be low in this case, the abnormal shake of the vehicle may not be caused by the second area, and the second area may not have actual abnormality.
For example, if the vehicle shake situation coincides with the size of the second region and the degree of concavity and convexity displayed in the image, it means that the abnormal shake of the vehicle is likely to be caused by the second region, and therefore the second region may be determined as a second-type road surface abnormal region.
Optionally, referring to fig. 4, before the abnormal shake feature is identified according to the continuous running image, the method further includes:
s31, identifying a second driving path of the vehicle according to the continuous driving images after the target image;
s32, identifying whether the second area is on a second driving path;
s33, if the second area is on the second running path, identifying abnormal shake features according to the continuous running images;
in this embodiment, whether the vehicle passes through the second area or not may be further identified according to the continuous running image, so that the association between the abnormal shake detected later and the second area is increased, and whether the second area is abnormal or not can be accurately determined by combining the influence of the running path of the vehicle and the second area. It can be understood that in the process, other sensing data or higher sensing precision is not needed, and the road surface abnormality which is not directly identified can be further detected through the vehicle running path and the image change of the vehicle recorder, so that the accuracy of road surface abnormality detection is improved.
Optionally, identifying the abnormal response feature from the continuous running image includes:
the vehicle speed is identified from the continuous travel image as decreasing by a fourth threshold within a predetermined distance from the second region.
The fourth threshold may be a fixed speed value, such as 20km/h,30km/h, etc.; taking 20km/h as an example, if the vehicle is decelerated to less than 60km/h during the approach to the second region, it is determined that the abnormal response characteristic is recognized, provided that the original running speed of the vehicle is about 80 km/h. At this time, it can be considered that the driver performs the corresponding response behavior based on the second region, and the second region is determined as the second-type road surface abnormality region. On the other hand, the fourth threshold may be a ratio, for example, 25%, 50%, or the like, and the abnormal response characteristic may be determined if the vehicle speed is reduced by the corresponding ratio as compared with the previous speed.
Optionally, the continuous running image within a predetermined time after the target image is acquired includes:
calculating the driving-off time of the vehicle from the second area according to the driving speed of the vehicle;
and acquiring continuous running images with the duration not less than the driving-off time.
For example, the driving speed of the vehicle may also be determined from the driving image, for example, from pixels of the reference object moving in the image frame. It should be understood that the travel speed may be expressed as a usual speed unit such as km/h, or may be expressed directly in terms of the pixel movement speed of the reference in the image. It can be seen that, in some examples, the pavement anomaly monitoring method of the present embodiment may complete detection based on the collected driving image entirely, and has low requirements for hardware, without deploying a large number of sensor devices. In other implementations, the driving speed of the vehicle may be obtained by an operating system of the vehicle, and the distance between the driving image and the second area is determined according to the driving image, so as to calculate the driving-away time.
The duration of the acquired continuous running images is determined based on the running speed of the vehicle, so that the running images before and after the vehicle runs to the second area are completely acquired, missing information is avoided, analysis processing of irrelevant images in some examples can be avoided, and resources are saved. For example, only the acquisition duration may be equal to the travel-off time, or the duration may be determined as the travel-off time plus a few seconds or the like, for example, plus 3S, 5S, or the like.
According to the pavement anomaly monitoring method, through further analysis and monitoring of the second area with the identification judgment probability between the first threshold value and the second threshold value, continuous running images in a preset time after the target image are obtained, and the second area in one target image before the continuous running image with the abnormal response characteristic is determined to be a second type pavement anomaly area after the abnormal response characteristic is identified; the road surface abnormality which cannot be directly identified is further monitored, the accuracy of road surface abnormality monitoring is improved, the road surface abnormality can be detected only by the running image acquired by the automobile data recorder in some implementation processes, a large number of sensing devices are not needed, and the cost is low and the feasibility is high.
Embodiment two:
the present embodiment provides a road surface abnormality monitoring system 100, referring to fig. 5, the road surface abnormality monitoring system 100 includes:
an image acquisition module 101, configured to acquire a driving image acquired by a driving recorder;
a first abnormality recognition module 102, configured to recognize a road surface abnormality region in a running image based on the running image; determining a first area with the identification judgment probability higher than a first threshold value as a first type of pavement abnormal area;
a second anomaly identification module 103, configured to determine, as a target image, a running image when each second region first appears, where the second region is a region where the identification decision probability is between a first threshold and a second threshold, and the second threshold is lower than the first threshold; acquiring continuous running images within a preset time after a target image is acquired; identifying abnormal response characteristics according to the continuous driving images; determining a second area in one target image before the continuous running image with the abnormal response characteristic as a second type of road surface abnormal area;
and the abnormality reporting module 104 is configured to report the determined type and position of each abnormal road surface area under the condition of permission of the user.
Alternatively, the second abnormality recognition module 103 recognizes the abnormality response feature from the continuous running image specifically including:
it is recognized that the vehicle has changed the travel path from the continuous travel image.
Optionally, the second abnormality recognition recognizes that the vehicle has changed the travel path based on the continuous travel image specifically includes:
identifying a first travel path of the vehicle according to a travel image in front of the target image;
identifying a second driving path of the vehicle according to the continuous driving images after the target image;
comparing the first travel path with the second travel path;
when the first travel path and the second travel path are different, it is determined that the vehicle has changed the travel path.
Optionally, the second abnormality recognition module 103 recognizing the abnormality response feature according to the continuous running image may further include:
identifying abnormal shake features according to the continuous running images;
according to the abnormal response characteristics, determining that the second area in the target image is the second type of road surface abnormal area comprises:
and determining the second area as a second type of road surface abnormal area according to the abnormal shake characteristics.
The second anomaly identification module 103 determines the second region as a second type of road surface anomaly region according to the anomaly shaking feature, including:
identifying abnormal features of the second region according to the target image;
judging the association degree of the abnormal shake features and the abnormal features of the second area, and determining the second area with the association degree higher than a third threshold value as a second type road surface abnormal area
Wherein the anomaly characteristic comprises at least one of:
the size of the second region;
degree of concavity and convexity of the second region.
The second anomaly identification module 103 further includes, before identifying the anomaly shaking feature from the continuous running image:
identifying a second driving path of the vehicle according to the continuous driving images after the target image;
identifying whether the second area is on a second travel path;
if the second area is on the second running path, identifying abnormal shake features according to the continuous running images.
Optionally, the second abnormality recognition module 103 recognizing the abnormality response feature according to the continuous running image may further include: the running speed of the vehicle is recognized to decrease by a fourth threshold value within a predetermined distance from the second region based on the continuous running image.
Alternatively, the continuous running image within a predetermined time after the second abnormality recognition module 103 acquires the target image includes: calculating the driving-off time of the vehicle from the second area according to the driving speed of the vehicle;
and acquiring continuous running images with the duration not less than the driving-off time.
In the pavement anomaly monitoring system 100 of the present embodiment, a second anomaly identification module 103 performs further analysis and monitoring on a second region with an identification decision probability between a first threshold value and a second threshold value, acquires continuous running images within a predetermined time after a target image, and determines that the second region in one target image before the continuous running image with the anomaly response feature appears after the anomaly response feature is identified as a second type pavement anomaly region; the road surface abnormality which is difficult to directly identify is further monitored, the accuracy of road surface abnormality monitoring is improved, the road surface abnormality can be identified only by the driving image acquired by the driving recorder in some implementation processes, a large number of sensing devices are not needed, and the cost is low and the feasibility is high.
It will be appreciated by those skilled in the art that the modules or steps of the embodiments of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a memory device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps in them may be fabricated into a single integrated circuit module for implementation. Thus, embodiments of the application are not limited to any specific combination of hardware and software.
The above description is only of the preferred embodiments of the present application and is not intended to limit the present application, and various modifications and variations can be made to the embodiments of the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.
Claims (6)
1. A pavement anomaly monitoring method, comprising:
acquiring a driving image acquired by a driving recorder;
identifying a road surface abnormal region in the driving image based on the driving image;
determining a first area with the identification judgment probability higher than a first threshold value as a first type of pavement abnormal area;
determining the running image when each second area appears for the first time as a target image, wherein the second area is an area with the identification judgment probability between the first threshold value and a second threshold value, the second threshold value is lower than the first threshold value, and the higher the identification judgment probability is, the higher the possibility of abnormality is;
acquiring continuous running images within a preset time after the target image;
identifying abnormal response characteristics according to the continuous driving images;
determining that the second region in the target image before the continuous running image with the abnormal response characteristic is a second type of road surface abnormal region;
reporting the determined type and position of each road surface abnormal area under the condition of user permission;
the identifying the abnormal response feature according to the continuous running image comprises the following steps:
identifying abnormal jitter characteristics according to the continuous driving images;
according to the abnormal response characteristics, determining that the second area in the target image is a second type of road surface abnormal area comprises:
determining the second area as the second type pavement abnormal area according to the abnormal jitter characteristics;
the determining that the second area is the second type of road surface abnormal area according to the abnormal shake feature includes:
identifying abnormal features of the second region according to the target image;
judging the association degree of the abnormal shaking characteristics and the abnormal characteristics of the second area, and determining the second area with the association degree higher than a third threshold value as the second type pavement abnormal area;
the abnormal feature includes at least one of a size of the second region and a degree of concavity and convexity of the second region;
before the abnormal jitter feature is identified according to the continuous running image, the method further comprises:
identifying a second driving path of the vehicle according to the continuous driving image after the target image;
identifying whether the second area is on the second travel path;
and if the second area is positioned on the second running path, identifying abnormal shake features according to the continuous running images.
2. The pavement anomaly monitoring method of claim 1, wherein the identifying an anomaly response feature from the continuous travel image includes:
and recognizing that the vehicle changes the driving path according to the continuous driving image.
3. The pavement anomaly monitoring method of claim 2, wherein the identifying that the vehicle has changed the travel path based on the continuous travel image includes:
identifying a first travel path of the vehicle according to a travel image in front of the target image;
identifying a second travel path of the vehicle according to the continuous travel image after the target image;
comparing the first travel path and the second travel path;
and when the first running path and the second running path are different, determining that the running path of the vehicle is changed.
4. The pavement anomaly monitoring method of claim 1, wherein the identifying an anomaly response feature from the continuous travel image includes:
and recognizing that the running speed of the vehicle is reduced by a fourth threshold value within a preset distance from the second area according to the continuous running images.
5. The pavement anomaly monitoring method of any one of claims 1-4, wherein the continuous travel image within a predetermined time after the acquisition of the target image comprises:
calculating the driving-off time of the vehicle from the second area according to the driving speed of the vehicle;
and acquiring continuous running images with the duration not less than the driving-off time.
6. A pavement anomaly monitoring system, comprising:
the image acquisition module is used for acquiring a driving image acquired by the driving recorder;
the first abnormality identification module is used for identifying a pavement abnormality area in the running image based on the running image; determining a first area with the identification judgment probability higher than a first threshold value as a first type of pavement abnormal area, wherein the higher the identification judgment probability is, the higher the possibility of abnormality is;
the second abnormality recognition module is used for determining the running image when each second area appears for the first time as a target image, wherein the second area is an area with recognition judgment probability between the first threshold value and a second threshold value, and the second threshold value is lower than the first threshold value; acquiring continuous running images within a preset time after the target image; identifying abnormal response characteristics according to the continuous driving images; determining that the second region in the target image before the continuous running image with the abnormal response characteristic is a second type of road surface abnormal region;
the abnormal reporting module is used for reporting the determined type and position of each pavement abnormal area under the condition of permission of a user;
the second abnormality recognition module recognizes an abnormality response feature from the continuous running image, including:
identifying abnormal jitter characteristics according to the continuous driving images;
according to the abnormal response characteristics, determining that the second area in the target image is a second type of road surface abnormal area comprises:
determining the second area as the second type pavement abnormal area according to the abnormal jitter characteristics;
the second anomaly identification module determines the second region as the second type of road surface anomaly region according to the anomaly shaking feature, including:
identifying abnormal features of the second region according to the target image;
judging the association degree of the abnormal shaking characteristics and the abnormal characteristics of the second area, and determining the second area with the association degree higher than a third threshold value as the second type pavement abnormal area;
the abnormal feature includes at least one of a size of the second region and a degree of concavity and convexity of the second region;
the second anomaly identification module further includes, before identifying the anomaly shaking feature according to the continuous running image:
identifying a second driving path of the vehicle according to the continuous driving image after the target image;
identifying whether the second area is on the second travel path;
and if the second area is positioned on the second running path, identifying abnormal shake features according to the continuous running images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310617963.5A CN116343176B (en) | 2023-05-30 | 2023-05-30 | Pavement abnormality monitoring system and monitoring method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310617963.5A CN116343176B (en) | 2023-05-30 | 2023-05-30 | Pavement abnormality monitoring system and monitoring method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116343176A CN116343176A (en) | 2023-06-27 |
| CN116343176B true CN116343176B (en) | 2023-08-11 |
Family
ID=86876296
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310617963.5A Active CN116343176B (en) | 2023-05-30 | 2023-05-30 | Pavement abnormality monitoring system and monitoring method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116343176B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110738081A (en) * | 2018-07-19 | 2020-01-31 | 杭州海康威视数字技术股份有限公司 | Abnormal road condition detection method and device |
| CN112163348A (en) * | 2020-10-24 | 2021-01-01 | 腾讯科技（深圳）有限公司 | Method and device for detecting road abnormal road surface, simulation method, vehicle and medium |
| CN112287910A (en) * | 2020-12-25 | 2021-01-29 | 智道网联科技(北京)有限公司 | Road abnormal area detection method, road abnormal area detection device, electronic equipment and storage medium |
| DE102019132012A1 (en) * | 2019-11-26 | 2021-05-27 | Connaught Electronics Ltd. | Method and system for the detection of small unclassified obstacles on a road surface |
| CN113361299A (en) * | 2020-03-03 | 2021-09-07 | 浙江宇视科技有限公司 | Abnormal parking detection method and device, storage medium and electronic equipment |
| JP2021152255A (en) * | 2020-03-24 | 2021-09-30 | 株式会社アイシン | Road surface abnormality follow-up observation device and computer program |
| WO2022070230A1 (en) * | 2020-09-29 | 2022-04-07 | 日本電気株式会社 | Road surface management device, road surface management method, terminal device, and recording medium |
| CN114299002A (en) * | 2021-12-24 | 2022-04-08 | 中用科技有限公司 | Intelligent detection system and method for abnormal road surface throwing behavior |
| CN114758322A (en) * | 2022-05-13 | 2022-07-15 | 安徽省路通公路工程检测有限公司 | Road quality detection system based on machine identification |
| CN114764973A (en) * | 2020-12-30 | 2022-07-19 | 华为技术有限公司 | Method, device and equipment for monitoring abnormal area of road surface and storage medium |
| CN114913449A (en) * | 2022-04-14 | 2022-08-16 | 深圳季连科技有限公司 | Road surface camera shooting analysis method for Internet of vehicles |
| WO2022195956A1 (en) * | 2021-03-17 | 2022-09-22 | 株式会社アイシン | System for detecting road surface abnormalities |
| CN115503747A (en) * | 2022-09-27 | 2022-12-23 | 吉林大学 | Road condition identification and reminding system based on intelligent automobile steer-by-wire system |
| WO2023042291A1 (en) * | 2021-09-15 | 2023-03-23 | 三菱電機株式会社 | Road surface anomaly determination system, vehicle-mounted device, and road surface anomaly determination method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2019028840A (en) * | 2017-08-01 | 2019-02-21 | 株式会社デンソー | Vehicle safety determination device, method and program |
-
2023
- 2023-05-30 CN CN202310617963.5A patent/CN116343176B/en active Active
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110738081A (en) * | 2018-07-19 | 2020-01-31 | 杭州海康威视数字技术股份有限公司 | Abnormal road condition detection method and device |
| DE102019132012A1 (en) * | 2019-11-26 | 2021-05-27 | Connaught Electronics Ltd. | Method and system for the detection of small unclassified obstacles on a road surface |
| CN113361299A (en) * | 2020-03-03 | 2021-09-07 | 浙江宇视科技有限公司 | Abnormal parking detection method and device, storage medium and electronic equipment |
| JP2021152255A (en) * | 2020-03-24 | 2021-09-30 | 株式会社アイシン | Road surface abnormality follow-up observation device and computer program |
| WO2022070230A1 (en) * | 2020-09-29 | 2022-04-07 | 日本電気株式会社 | Road surface management device, road surface management method, terminal device, and recording medium |
| CN112163348A (en) * | 2020-10-24 | 2021-01-01 | 腾讯科技（深圳）有限公司 | Method and device for detecting road abnormal road surface, simulation method, vehicle and medium |
| WO2022083409A1 (en) * | 2020-10-24 | 2022-04-28 | 腾讯科技（深圳）有限公司 | Detection method and simulation method for abnormal road surface of road, and related apparatus |
| CN112287910A (en) * | 2020-12-25 | 2021-01-29 | 智道网联科技(北京)有限公司 | Road abnormal area detection method, road abnormal area detection device, electronic equipment and storage medium |
| CN114764973A (en) * | 2020-12-30 | 2022-07-19 | 华为技术有限公司 | Method, device and equipment for monitoring abnormal area of road surface and storage medium |
| WO2022195956A1 (en) * | 2021-03-17 | 2022-09-22 | 株式会社アイシン | System for detecting road surface abnormalities |
| WO2023042291A1 (en) * | 2021-09-15 | 2023-03-23 | 三菱電機株式会社 | Road surface anomaly determination system, vehicle-mounted device, and road surface anomaly determination method |
| CN114299002A (en) * | 2021-12-24 | 2022-04-08 | 中用科技有限公司 | Intelligent detection system and method for abnormal road surface throwing behavior |
| CN114913449A (en) * | 2022-04-14 | 2022-08-16 | 深圳季连科技有限公司 | Road surface camera shooting analysis method for Internet of vehicles |
| CN114758322A (en) * | 2022-05-13 | 2022-07-15 | 安徽省路通公路工程检测有限公司 | Road quality detection system based on machine identification |
| CN115503747A (en) * | 2022-09-27 | 2022-12-23 | 吉林大学 | Road condition identification and reminding system based on intelligent automobile steer-by-wire system |
Non-Patent Citations (1)
| Title |
|---|
| 基于图像处理的人员异常行为监测设计;王帅鹏;赵凯;;现代电子技术(06);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116343176A (en) | 2023-06-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102428505B (en) | Vehicular Environment Estimation Device | |
| US11836985B2 (en) | Identifying suspicious entities using autonomous vehicles | |
| CN107527511B (en) | Intelligent vehicle driving reminding method and device | |
| JP2016095831A (en) | Driving support system and center | |
| CN109712401B (en) | Composite road network bottleneck point identification method based on floating car track data | |
| CN111753612B (en) | Method and device for detecting casting object and storage medium | |
| US20200117950A1 (en) | System and method for evaluating a trained vehicle data set familiarity of a driver assitance system | |
| CN109703456B (en) | Warning method and device for preventing automobile collision and automobile controller | |
| CN112749210B (en) | Vehicle collision recognition method and system based on deep learning | |
| CN113127466B (en) | Vehicle track data preprocessing method and computer storage medium | |
| CN112242058B (en) | Target abnormity detection method and device based on traffic monitoring video and storage medium | |
| US20220254249A1 (en) | Traffic event and road condition identification and classification | |
| CN106650730A (en) | Turn signal lamp detection method and system in car lane change process | |
| CN113269042A (en) | Intelligent traffic management method and system based on running vehicle violation identification | |
| KR101628547B1 (en) | Apparatus and Method for Checking of Driving Load | |
| CN109887303B (en) | Lane-changing behavior early warning system and method | |
| JP5997962B2 (en) | In-vehicle lane marker recognition device | |
| CN116343176B (en) | Pavement abnormality monitoring system and monitoring method thereof | |
| US10415981B2 (en) | Anomaly estimation apparatus and display apparatus | |
| KR102228559B1 (en) | Method and system for seinsing fatigue state of driver | |
| CN116564083A (en) | Expressway traffic jam detection method based on improved CrowdDet algorithm | |
| CN112990117B (en) | Installation data processing method and device based on intelligent driving system | |
| CN114494938A (en) | Non-motor vehicle behavior identification method and related device | |
| JP2018073049A (en) | Image recognition device, image recognition system, and image recognition method | |
| JP2005176077A (en) | Camera monitoring system and its monitoring control method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |