CN116229231A - Bridge ship collision detection method and system based on deep learning algorithm - Google Patents
Bridge ship collision detection method and system based on deep learning algorithm Download PDFInfo
- Publication number
- CN116229231A CN116229231A CN202310108876.7A CN202310108876A CN116229231A CN 116229231 A CN116229231 A CN 116229231A CN 202310108876 A CN202310108876 A CN 202310108876A CN 116229231 A CN116229231 A CN 116229231A
- Authority
- CN
- China
- Prior art keywords
- ship
- bridge
- collision
- target frames
- areas
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 69
- 238000013135 deep learning Methods 0.000 title claims abstract description 35
- 238000012544 monitoring process Methods 0.000 claims abstract description 33
- 238000000034 method Methods 0.000 claims abstract description 15
- 238000012549 training Methods 0.000 claims description 18
- 238000009434 installation Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 claims description 9
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 claims description 9
- 230000008034 disappearance Effects 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 abstract description 4
- 238000003384 imaging method Methods 0.000 abstract description 4
- 238000012423 maintenance Methods 0.000 description 6
- 238000007726 management method Methods 0.000 description 4
- 238000013136 deep learning model Methods 0.000 description 3
- 238000011897 real-time detection Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- NKYAAYKKNSYIIW-XVFCMESISA-N 5-aminoimidazole ribonucleoside Chemical compound NC1=CN=CN1[C@H]1[C@H](O)[C@H](O)[C@@H](CO)O1 NKYAAYKKNSYIIW-XVFCMESISA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- QZXCCPZJCKEPSA-UHFFFAOYSA-N chlorfenac Chemical compound OC(=O)CC1=C(Cl)C=CC(Cl)=C1Cl QZXCCPZJCKEPSA-UHFFFAOYSA-N 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013210 evaluation model Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- LFULEKSKNZEWOE-UHFFFAOYSA-N propanil Chemical compound CCC(=O)NC1=CC=C(Cl)C(Cl)=C1 LFULEKSKNZEWOE-UHFFFAOYSA-N 0.000 description 1
- 239000003643 water by type Substances 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/08—Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
- G06Q10/083—Shipping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20104—Interactive definition of region of interest [ROI]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A30/00—Adapting or protecting infrastructure or their operation
- Y02A30/30—Adapting or protecting infrastructure or their operation in transportation, e.g. on roads, waterways or railways
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Economics (AREA)
- Development Economics (AREA)
- General Business, Economics & Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Geometry (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a bridge ship collision detection method and a system based on a deep learning algorithm, wherein an imaging device is arranged on the shore of a bridge to be detected, a plurality of ship running pictures in a bridge monitoring area are acquired through the imaging device, the position coordinates of all ships in each picture are obtained by adopting a Yolov5 model, the ID numbers of all ships are obtained by adopting an IOU tracking algorithm, meanwhile, the ship running track is analyzed in real time by adopting a specific collision judgment method, and whether the ship collides with a pier is judged, and once the track is abnormal, video picture evidence can be immediately alarmed and stored.
Description
Technical Field
The invention relates to the technical field of bridge protection, in particular to a method and a system for detecting collision of a bridge and a ship based on a deep learning algorithm.
Background
With the development of water transportation, the frequent occurrence of the collision event of the bridge brings great loss to traffic authorities, and the safety of the bridge foundation is one of the forever important subjects in the bridge protection industry of the large bridge. The existing maintenance and management method for the urban bridge is mainly used for arranging daily inspection and maintenance of specific workers, so that the cost of manpower and material resources is high, the efficiency is insufficient, long-time supervision is not realized, and loopholes are easy to appear. Especially when the ship collides with the bridge, the ship has a small driving distance within a certain time range, namely, the ship is in a static or creep state for a period of time and immediately drives off the site, so that the manager can hardly save accident evidence and timely drive to the site for treatment.
The existing collision early warning schemes mainly have two kinds: 1) By means of a big data analysis technology, a large amount of data such as AIS, DCPA and TCPA of ships are analyzed, and a bridge area water area ship collision risk evaluation model is built. 2) Through setting up equipment such as camera and laser range finder on the bridge once more, through the data fusion analysis of laser and video, monitor the navigation state in the waters of landing stage bridge both sides, this scheme hardware installation and maintenance's cost are too high.
Disclosure of Invention
In order to solve the problems of large workload, low universality, over-high installation and maintenance cost and the like in the existing ship collision early warning process, the invention provides a bridge ship collision detection method based on a deep learning algorithm, which adopts a Yolov5 model, an IOU tracking algorithm and a collision judgment method and is mutually combined with real-time detection and tracking of the navigation state of a ship, and early warning can be sent out on a collision event when abnormal stay of the ship under a bridge is found, so that the discovery of accidents or potential hidden hazards by workers is effectively accelerated, the labor cost is saved, the detection efficiency is increased, and the safety of the bridge is improved. The invention further relates to a bridge ship collision detection system based on the deep learning algorithm.
The technical scheme of the invention is as follows:
the bridge ship collision detection method based on the deep learning algorithm is characterized by comprising the following steps of:
an image detection step: setting a camera device on the bank of a specific distance threshold from the bridge to be detected, and acquiring a plurality of ship running pictures in a monitoring video stream in a bridge monitoring area through the camera device;
position detection: calibrating all ship positions in each ship running picture, constructing a training data set according to all ship running pictures after calibration, inputting the training data set into a Yolov5 model, training the Yolov5 model based on a deep learning algorithm to obtain a ship detection model, obtaining the position coordinates of all ships in each picture according to the trained ship detection model, and further obtaining the position coordinates of the diagonal vertexes of the target frames of each ship in each ship running picture;
ID number determining step: respectively calculating the areas of the two target frames and the areas of the intersecting parts of the two target frames according to the diagonal vertex position coordinates of the respective target frames, calculating the IOU values of the two target frames according to the areas of the two target frames and the areas of the intersecting parts of the two target frames by adopting an IOU tracking algorithm, comparing the IOU values with a preset numerical threshold, and judging that the ships in the two target frames belong to the same ship and have the same ID number if the IOU values of the two target frames are larger than the preset first numerical threshold;
and a collision judging step: and repeating the ID number determining step to obtain the ID numbers of all the ships, calibrating all pier areas in a bridge monitoring area on each ship driving picture as ship collision occurrence areas, storing the position coordinates of all the ships in each frame of ship driving picture in a monitoring video stream in a certain time period and the ID numbers corresponding to each ship into a history queue, traversing the position coordinates of all the ships in the history queue, and judging that the ship collides with the piers and alarming if the center point coordinates of the target frame of a certain ship are positioned in the ship collision occurrence areas and maintained for a certain period of time, and the IOU values of the target frame in the ship driving picture of the current frame and the target frame in the ship driving picture of the previous frame are larger than a preset second numerical threshold and maintained for a certain period of time.
Preferably, in the image detection step, the bridge monitoring area includes a bridge hole of the bridge and a water surface located below the bridge hole.
Preferably, the camera device is arranged on the bank side with the distance from the bridge to be detected being 50-100 m, and the installation height is 6 m.
Preferably, in the step of determining the ID number, the IOU tracking algorithm calculates the IOU values of the two target frames based on the intersection ratio of the two target frames.
Preferably, the collision judging step obtains a collision record after judging that the ship collides with the pier, and the collision record comprises three state diagrams of the occurrence, collision and disappearance of the ship.
A bridge ship collision detection system based on a deep learning algorithm is characterized by comprising an image detection module, a position detection module, an ID number determination module and a collision judgment module which are connected in sequence,
the image detection module comprises a camera device arranged on the bank side of a specific distance threshold value from the bridge to be detected, and a plurality of ship running pictures in a monitoring video stream in a bridge monitoring area are acquired through the camera device;
the position detection module is used for calibrating all ship positions in each ship running picture, constructing a training data set according to all ship running pictures after calibration, inputting the training data set into a Yolov5 model, training the Yolov5 model based on a deep learning algorithm to obtain a ship detection model, obtaining the position coordinates of all ships in each picture according to the trained ship detection model, and further obtaining the diagonal vertex position coordinates of a target frame of each ship in each ship running picture;
the method comprises the steps that an ID number determining module calculates the areas of two target frames and the areas of intersecting parts of the two target frames according to diagonal vertex position coordinates of the two target frames in two continuous ship running pictures, calculates IOU values of the two target frames according to the areas of the two target frames and the areas of the intersecting parts of the two target frames by adopting an IOU tracking algorithm, compares the IOU values with a preset numerical threshold, and judges that ships in the two target frames belong to the same ship and have the same ID number if the IOU values of the two target frames are larger than the preset first numerical threshold; further obtaining the ID numbers of all ships;
the collision judging module is used for calibrating all pier areas in a bridge monitoring area on each ship running picture as ship collision occurrence areas, storing position coordinates of all ships in each frame of ship running picture in a monitoring video stream in a certain time period and ID numbers corresponding to each ship into a history queue, traversing the position coordinates of all ships in the history queue, and judging that the ship collides with the piers and gives an alarm if the center point coordinates of the target frame of a certain ship are positioned in the ship collision occurrence areas and maintained for a certain period of time, and the IOU value of the target frame in the ship running picture of the current frame and the target frame in the ship running picture of the previous frame is greater than a preset second numerical threshold and maintained for a certain period of time.
Preferably, the camera device adopts a spherical camera, the position detection module, the ID number determination module and the collision judgment module are integrated in the industrial personal computer, and the spherical camera is directly or indirectly connected with the industrial personal computer through a router.
Preferably, in the image detection module, the bridge monitoring area includes a bridge hole of the bridge and a water surface located below the bridge hole.
Preferably, the camera device is arranged on the bank side with the distance from the bridge to be detected being 50-100 m, and the installation height is 6 m.
Preferably, the collision judgment module obtains a collision record after judging that the ship collides with the pier, wherein the collision record comprises three state diagrams of the occurrence, collision and disappearance of the ship.
The invention has the following technical effects:
the invention provides a bridge ship collision detection method based on a deep learning algorithm, wherein a camera device is arranged on the bank side of a specific distance threshold from a bridge to be detected, a plurality of ship running pictures in a monitoring video stream in a bridge monitoring area (simply referred to as a monitoring area) are acquired through the camera device, only one camera device is needed to be erected on the bank side, equipment is not needed to be erected below the bridge, later equipment maintenance is also carried out on the bank side, and hardware deployment and maintenance are convenient; the position coordinates of all ships in each picture are obtained by adopting a Yolov5 model (also called as a deep learning model Yolov 5) based on a deep learning algorithm, and the ID numbers of all ships are obtained by adopting an IOU tracking algorithm, so that the ships appearing in continuous pictures can be tracked, the running track of each ship from appearing to disappearing in the pictures is obtained, and the ship can be directly transplanted into the application environment of other bridges, so that the universality is high; meanwhile, a specific collision judgment method is adopted to analyze the ship running track in real time, and once the track is abnormal or collision occurs, the alarm is immediately given out and video picture evidence is stored, so that the real-time performance is high. The invention can greatly accelerate the discovery of accidents or potential hidden troubles by staff, reduce the time for reaching the site to solve the problems, greatly increase the detection efficiency and improve the safety of bridges while saving the labor cost.
The invention also relates to a bridge ship collision detection system based on the deep learning algorithm, which corresponds to the bridge ship collision detection method based on the deep learning algorithm, and comprises an image detection module, a position detection module, an ID number determination module and a collision judgment module which are sequentially connected, wherein the modules work cooperatively, the navigation state of the ship is detected and tracked in real time by adopting the deep learning algorithm of the Yolov5 model, the IOU tracking algorithm and the collision judgment method and combining the real-time detection and tracking, and when abnormal stay of the ship under the bridge is found, an early warning can be sent out for a collision event, so that the discovery of accidents or potential hidden hazards by workers is effectively accelerated, the labor cost is saved, the detection efficiency is increased, and the safety of the bridge is improved.
Drawings
Fig. 1 is a flowchart of a bridge ship collision detection method based on a deep learning algorithm.
Fig. 2 is a schematic diagram of a hardware installation scheme of the bridge ship collision detection method based on the deep learning algorithm.
Fig. 3 is a position coordinate diagram of a ship in a ship running picture according to the present invention.
Fig. 4 is a schematic view of a pier region according to the present invention.
Fig. 5-7 are three state diagrams of bridge vessel crash records, respectively.
Detailed Description
The present invention will be described below with reference to the accompanying drawings.
When a ship collides with a bridge, the ship has smaller driving distance within a certain time range, namely the ship is in a static or slow running state for a period of time, according to the rule, the invention provides a bridge ship collision detection method based on a deep learning algorithm, video data of a detection scene are collected through a camera device such as a high-definition camera erected on the shore, and the detection scene is trained by using a Yolov5 model (namely using the deep learning model Yolov 5) based on the deep learning algorithm, so that the functions of accurately detecting various ships in a picture are achieved; and tracking the ships appearing in the continuous pictures by using an IOU tracking algorithm to obtain the running track of each ship from appearing to disappearing under the pictures. And calibrating the area where the bridge under the picture of the camera and the water area below the bridge are located, analyzing the relationship between the coordinates of the track points of the ship and the area in real time, judging that the ship collides under the bridge if the track points are in the area for a long time, storing collision pictures and videos, and reporting to the management terminal in real time. The flow chart of the method is shown in fig. 1, and includes an image detection step, a position detection step, an ID number determination step, and a collision judgment step.
An image detection step: arranging an imaging device on the shore of a bridge to be detected, and acquiring a plurality of ship running pictures in a monitoring area through the imaging device;
specifically, as shown in the hardware installation scheme in fig. 2, a camera device is installed on the shore 50-100 m away from the bridge to be detected, the installation height is preferably about 6 m (the installation height can be adjusted and set adaptively), and the field of view of the camera device, namely the bridge monitoring area, covers the bridge hole of the bridge and the water surface below the bridge hole; wherein, camera device includes the dome camera. The following position detection step, ID number determination step and collision judgment step are executed by an industrial personal computer, the spherical camera and the industrial personal computer can be directly connected, or as shown in fig. 2, the spherical camera (30 times of high-definition ball machine) is connected with the industrial personal computer through a 4G router, namely, the 30 times of high-definition ball machine, the 4G router and the industrial personal computer are sequentially connected, the industrial personal computer integrates the following three steps for processing and analyzing the video stream acquired in real time, judging whether collision exists or not, judging whether an alarm exists or not, and the 4G router is connected with the industrial personal computer and used for transmitting an alarm signal to a remote management terminal, so that a manager can process a collision event in time. The preferred configuration list of the hardware device is shown in table 1.
Table 1:
position detection: as shown in fig. 3, manually calibrating all ship positions in each ship running picture, constructing a training data set according to all ship running pictures after calibration, inputting the training data set into a Yolov5 model, training the Yolov5 model based on a deep learning algorithm, namely obtaining a ship detection model through the deep learning model Yolov5, obtaining position coordinates Loc of all ships in each picture according to the trained ship detection model, and further obtaining target frame diagonal vertex position coordinates Loc { (x) of each ship in each ship running picture 1 ,y 1 ),(x 2 ,y 2 )}；
An ID number determining step, namely respectively calculating the areas of two target frames and the areas of the intersecting parts of the two target frames according to the diagonal vertex position coordinates of the respective target frames by using the target frames of two corresponding vessels in the continuous two-frame ship running pictures, calculating the IOU values of the two target frames according to the areas of the two target frames and the areas of the intersecting parts of the two target frames by adopting an IOU tracking algorithm, comparing the IOU values with a preset numerical threshold, and judging that the vessels in the two target frames belong to the same vessel and have the same ID number if the IOU values of the two target frames are larger than the preset first numerical threshold;
specifically, the target frames of two vessels appearing correspondingly in the running pictures of two continuous vessels are firstly processed according to the diagonal vertex position coordinates Loc of the respective target frames 1 {(x 11 ,y 11 ),(x 21 ,y 21 ) ' and Loc 2 {(x 12 ,y 12 ),(x 22 ,y 22 ) Respectively calculating the areas Area of the two target frames 1 And Area 2 And the Area of the intersection of two target frames inter According to the area of the two target frames and the area of the intersection part of the two target frames, the IOU value of the two target frames is calculated by adopting an IOU tracking algorithm, namely the IOU tracking algorithm is based on the two target frames Loc 1 {(x 11 ,y 11 ),(x 21 ,y 21 ) ' and Loc 2 {(x 12 ,y 12 ),(x 22 ,y 22 ) And the cross ratio of the two is obtained. The IOU value is calculated according to the following formula:
wherein, area 1 And Area 2 Respectively Loc 1 And Loc 2 Area of Area (2) inter Is the area of the intersection of two target boxes.
Area of intersection of two target frames inter The calculation is performed according to the following formula:
Area inter ＝[min(x 21 ,x 22 )-max(x 11 ,x 12 )]×[min(y 21 ,y 22 )-max(y 11 ,y 12 )] (2)
comparing the IOU value with a preset numerical threshold, if the IOU value of two target frames is larger than a preset first numerical threshold, that is, the IOU value is preferably larger than 0.8, in two continuous frames, it can be judged that the ships in the two target frames belong to the same ship and have the same ID number.
And a collision judging step: as shown in fig. 4, the ID number determining step is repeated to obtain ID numbers of all ships, and all pier areas in the bridge monitoring area on each ship running picture are marked as ship collision occurrence areas for judging collision events, wherein 1-3 marked places in the pier areas are three places under the piers, which are easy to collide by the ship. And then, the position coordinates of all the ships in each frame of ship running picture in the monitoring video stream within a certain time period and the ID numbers corresponding to each ship are stored into a history queue, the position coordinates of all the ships in the history queue are traversed, if the center point coordinates of the target frame of a certain ship are positioned in a ship collision occurrence area and maintained for a certain period of time, such as 3 seconds or more, and the IOU value of the target frame in the ship running picture of the current frame and the target frame in the ship running picture of the previous frame of the current frame is greater than a preset second numerical threshold, namely preferably greater than 0.96, maintained for a certain period of time, such as 3 seconds or more, the ship is judged to collide with a pier, the collision picture and video are saved, and the management terminal is reported in real time and an alarm is given. Further, after judging that the ship collides with the pier, obtaining a collision record, wherein the collision record can comprise three state diagrams of recording the occurrence, collision and disappearance of the ship, as shown in fig. 5-7, namely the occurrence of the ship, the first occurrence of the ship in a picture, the collision of the ship with the pier and the disappearance of the ship, and the disappearance of the ship in the picture.
The invention also relates to a bridge ship collision detection system based on the deep learning algorithm, which comprises an image detection module, a position detection module, an ID number determination module and a collision judgment module which are connected in sequence,
the image detection module comprises a camera device, wherein the camera device is arranged on the bank side of a specific distance threshold from a bridge to be detected, and a plurality of ship running pictures in a monitoring video stream in a bridge monitoring area are acquired through the camera device;
the position detection module is used for calibrating all ship positions in each ship running picture, constructing a training data set according to all ship running pictures after calibration, inputting the training data set into a Yolov5 model, training the Yolov5 model based on a deep learning algorithm to obtain a ship detection model, obtaining the position coordinates of all ships in each picture according to the trained ship detection model, and further obtaining the diagonal vertex position coordinates of a target frame of each ship in each ship running picture;
the method comprises the steps that an ID number determining module calculates the areas of two target frames and the areas of intersecting parts of the two target frames according to diagonal vertex position coordinates of the two target frames in two continuous ship running pictures, calculates IOU values of the two target frames according to the areas of the two target frames and the areas of the intersecting parts of the two target frames by adopting an IOU tracking algorithm, compares the IOU values with a preset numerical threshold, and judges that ships in the two target frames belong to the same ship and have the same ID number if the IOU values of the two target frames are larger than the preset first numerical threshold; further obtaining the ID numbers of all ships;
the collision judging module is used for calibrating all pier areas in a bridge monitoring area on each ship running picture as ship collision occurrence areas, storing position coordinates of all ships in each frame of ship running picture in a monitoring video stream in a certain time period and ID numbers corresponding to each ship into a history queue, traversing the position coordinates of all ships in the history queue, and judging that the ship collides with the piers and gives an alarm if the center point coordinates of the target frame of a certain ship are positioned in the ship collision occurrence areas and maintained for a certain period of time, and the IOU value of the target frame in the ship running picture of the current frame and the target frame in the ship running picture of the previous frame is greater than a preset second numerical threshold and maintained for a certain period of time.
Preferably, referring to fig. 2, the camera device adopts a dome camera, the position detection module, the ID number determination module and the collision judgment module are integrated in the industrial personal computer, and the dome camera is directly or indirectly connected with the industrial personal computer through a router.
Preferably, the bridge monitoring area comprises a bridge opening of the bridge and a water surface below the bridge opening.
Preferably, the camera device is arranged on the bank side with the distance from the bridge to be detected being 50-100 m, and the installation height is 6 m.
Preferably, the collision judgment module obtains a collision record after judging that the ship collides with the pier, wherein the collision record comprises three state diagrams of the occurrence, collision and disappearance of the ship.
The invention provides an objective and scientific bridge ship collision detection method and system based on a deep learning algorithm, which are used for detecting and tracking the ship navigation state in real time by adopting a Yolov5 model, an IOU tracking algorithm and a collision judgment method in combination, and sending out early warning on a collision event when the ship is found to stay abnormally under a bridge, so that the accident or potential hidden danger is found by workers effectively quickened, the labor cost is saved, the detection efficiency is increased, and the safety of the bridge is improved.
It should be noted that the above-described embodiments will enable those skilled in the art to more fully understand the invention, but do not limit it in any way. Therefore, although the present invention has been described in detail with reference to the drawings and examples, it will be understood by those skilled in the art that the present invention may be modified or equivalent, and in all cases, all technical solutions and modifications which do not depart from the spirit and scope of the present invention are intended to be included in the scope of the present invention.
Claims (10)
1. The bridge ship collision detection method based on the deep learning algorithm is characterized by comprising the following steps of:
an image detection step: setting a camera device on the bank of a specific distance threshold from the bridge to be detected, and acquiring a plurality of ship running pictures in a monitoring video stream in a bridge monitoring area through the camera device;
position detection: calibrating all ship positions in each ship running picture, constructing a training data set according to all ship running pictures after calibration, inputting the training data set into a Yolov5 model, training the Yolov5 model based on a deep learning algorithm to obtain a ship detection model, obtaining the position coordinates of all ships in each picture according to the trained ship detection model, and further obtaining the position coordinates of the diagonal vertexes of the target frames of each ship in each ship running picture;
ID number determining step: respectively calculating the areas of the two target frames and the areas of the intersecting parts of the two target frames according to the diagonal vertex position coordinates of the respective target frames, calculating the IOU values of the two target frames according to the areas of the two target frames and the areas of the intersecting parts of the two target frames by adopting an IOU tracking algorithm, comparing the IOU values with a preset numerical threshold, and judging that the ships in the two target frames belong to the same ship and have the same ID number if the IOU values of the two target frames are larger than the preset first numerical threshold;
and a collision judging step: and repeating the ID number determining step to obtain the ID numbers of all the ships, calibrating all pier areas in a bridge monitoring area on each ship driving picture as ship collision occurrence areas, storing the position coordinates of all the ships in each frame of ship driving picture in a monitoring video stream in a certain time period and the ID numbers corresponding to each ship into a history queue, traversing the position coordinates of all the ships in the history queue, and judging that the ship collides with the piers and alarming if the center point coordinates of the target frame of a certain ship are positioned in the ship collision occurrence areas and maintained for a certain period of time, and the IOU values of the target frame in the ship driving picture of the current frame and the target frame in the ship driving picture of the previous frame are larger than a preset second numerical threshold and maintained for a certain period of time.
2. The method for detecting the collision of the bridge vessel based on the deep learning algorithm according to claim 1, wherein in the image detection step, the bridge monitoring area includes a bridge hole of the bridge and a water surface located below the bridge hole.
3. The method for detecting the collision of the bridge and the ship based on the deep learning algorithm according to claim 1, wherein the camera device is arranged on the bank side with the distance from the bridge to be detected being 50-100 meters, and the installation height is 6 meters.
4. The bridge vessel collision detection method based on the deep learning algorithm according to any one of claims 1 to 3, wherein in the ID number determination step, the IOU tracking algorithm calculates the IOU values of the two target frames based on the intersection ratio of the two target frames.
5. The method for detecting the collision of the bridge and the vessel based on the deep learning algorithm according to claim 4, wherein the collision judging step obtains a collision record after judging that the vessel collides with the pier, and the collision record comprises three state diagrams of the occurrence, the collision and the disappearance of the vessel.
6. A bridge ship collision detection system based on a deep learning algorithm is characterized by comprising an image detection module, a position detection module, an ID number determination module and a collision judgment module which are connected in sequence,
the image detection module comprises a camera device arranged on the bank side of a specific distance threshold value from the bridge to be detected, and a plurality of ship running pictures in a monitoring video stream in a bridge monitoring area are acquired through the camera device;
the position detection module is used for calibrating all ship positions in each ship running picture, constructing a training data set according to all ship running pictures after calibration, inputting the training data set into a Yolov5 model, training the Yolov5 model based on a deep learning algorithm to obtain a ship detection model, obtaining the position coordinates of all ships in each picture according to the trained ship detection model, and further obtaining the diagonal vertex position coordinates of a target frame of each ship in each ship running picture;
the method comprises the steps that an ID number determining module calculates the areas of two target frames and the areas of intersecting parts of the two target frames according to diagonal vertex position coordinates of the two target frames in two continuous ship running pictures, calculates IOU values of the two target frames according to the areas of the two target frames and the areas of the intersecting parts of the two target frames by adopting an IOU tracking algorithm, compares the IOU values with a preset numerical threshold, and judges that ships in the two target frames belong to the same ship and have the same ID number if the IOU values of the two target frames are larger than the preset first numerical threshold; further obtaining the ID numbers of all ships;
the collision judging module is used for calibrating all pier areas in a bridge monitoring area on each ship running picture as ship collision occurrence areas, storing position coordinates of all ships in each frame of ship running picture in a monitoring video stream in a certain time period and ID numbers corresponding to each ship into a history queue, traversing the position coordinates of all ships in the history queue, and judging that the ship collides with the piers and gives an alarm if the center point coordinates of the target frame of a certain ship are positioned in the ship collision occurrence areas and maintained for a certain period of time, and the IOU value of the target frame in the ship running picture of the current frame and the target frame in the ship running picture of the previous frame is greater than a preset second numerical threshold and maintained for a certain period of time.
7. The bridge ship collision detection system based on the deep learning algorithm according to claim 6, wherein the camera device adopts a dome camera, the position detection module, the ID number determination module and the collision judgment module are integrated in an industrial personal computer, and the dome camera is directly or indirectly connected with the industrial personal computer through a router.
8. The system for detecting the collision of the bridge and the ship based on the deep learning algorithm according to claim 6, wherein in the image detection module, the bridge monitoring area comprises a bridge hole of the bridge and a water surface below the bridge hole.
9. The bridge ship collision detection system based on the deep learning algorithm according to claim 6, wherein the camera device is arranged on the shore with a distance of 50-100 meters from the bridge to be detected, and the installation height is 6 meters.
10. The bridge and vessel collision detection system based on the deep learning algorithm according to one of claims 6 to 9, wherein the collision judgment module obtains a collision record after judging that the vessel collides with the pier, and the collision record comprises three state diagrams of recording the occurrence, collision and disappearance of the vessel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310108876.7A CN116229231A (en) | 2023-02-10 | 2023-02-10 | Bridge ship collision detection method and system based on deep learning algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310108876.7A CN116229231A (en) | 2023-02-10 | 2023-02-10 | Bridge ship collision detection method and system based on deep learning algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116229231A true CN116229231A (en) | 2023-06-06 |
Family
ID=86578106
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310108876.7A Pending CN116229231A (en) | 2023-02-10 | 2023-02-10 | Bridge ship collision detection method and system based on deep learning algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116229231A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116819025A (en) * | 2023-07-03 | 2023-09-29 | 中国水利水电科学研究院 | Water quality monitoring system and method based on Internet of things |
-
2023
- 2023-02-10 CN CN202310108876.7A patent/CN116229231A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116819025A (en) * | 2023-07-03 | 2023-09-29 | 中国水利水电科学研究院 | Water quality monitoring system and method based on Internet of things |
| CN116819025B (en) * | 2023-07-03 | 2024-01-23 | 中国水利水电科学研究院 | Water quality monitoring system and method based on Internet of things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111144291B (en) | Video monitoring area personnel intrusion discrimination method and device based on target detection | |
| KR101748121B1 (en) | System and method for detecting image in real-time based on object recognition | |
| CN103456024B (en) | A kind of moving target gets over line determination methods | |
| KR101081861B1 (en) | Violence detection method by analyzing the motion image of moving peoples | |
| JPWO2017047687A1 (en) | Monitoring system | |
| CN116229231A (en) | Bridge ship collision detection method and system based on deep learning algorithm | |
| KR20100119476A (en) | An outomatic sensing system for traffic accident and method thereof | |
| CN104966304A (en) | Kalman filtering and nonparametric background model-based multi-target detection tracking method | |
| CN109448365A (en) | Across the scale space base land regions road traffic system integrated supervision method of one kind | |
| CN111225189A (en) | Middle and small-sized channel bridge monitoring device | |
| CN111445726A (en) | Bridge anti-collision early warning system and method based on distributed video monitoring | |
| CN116208745A (en) | Dock ship berthing monitoring and recording system and method | |
| CN104866827A (en) | Method for detecting people crossing behavior based on video monitoring platform | |
| Ketcham et al. | The intruder detection system for rapid transit using CCTV surveillance based on histogram shapes | |
| CN112530144B (en) | Method and system for warning violation behaviors of thermal power plant based on neural network | |
| CN116453276A (en) | Marine wind power electronic fence monitoring and early warning method and system | |
| CN105405297B (en) | A kind of automatic detection method for traffic accident based on monitor video | |
| CN114401387A (en) | System and method for monitoring foreign matter invasion of rail transit line | |
| CN217061202U (en) | Bridge anticollision early warning system | |
| CN213424017U (en) | Intelligent construction site system | |
| CN112880733B (en) | Height limiting protective frame state monitoring device and method | |
| CN117351781A (en) | Active anti-collision early warning system and method for cross-sea bridge | |
| CN115019463B (en) | Water area supervision system based on artificial intelligence technology | |
| CN116597394A (en) | Railway foreign matter intrusion detection system and method based on deep learning | |
| CN116797967A (en) | Visual monitoring hidden trouble identification method and device for overhead transmission line |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |