CN117115765A - Fishing boat arrival and departure supervision method and system based on vision - Google Patents
Fishing boat arrival and departure supervision method and system based on vision Download PDFInfo
- Publication number
- CN117115765A CN117115765A CN202311329900.6A CN202311329900A CN117115765A CN 117115765 A CN117115765 A CN 117115765A CN 202311329900 A CN202311329900 A CN 202311329900A CN 117115765 A CN117115765 A CN 117115765A
- Authority
- CN
- China
- Prior art keywords
- fishing
- personnel
- fishing boat
- sea
- ship
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 34
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 111
- 238000001514 detection method Methods 0.000 claims abstract description 78
- 238000007635 classification algorithm Methods 0.000 claims abstract description 30
- 238000012552 review Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims description 22
- 238000013135 deep learning Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 15
- 238000004519 manufacturing process Methods 0.000 claims description 13
- 238000012544 monitoring process Methods 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 12
- 102100032202 Cornulin Human genes 0.000 claims description 4
- 101000920981 Homo sapiens Cornulin Proteins 0.000 claims description 4
- 238000012937 correction Methods 0.000 claims description 4
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 230000007306 turnover Effects 0.000 claims 1
- 230000006870 function Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
- G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image recognition, in particular to a method and a system for supervising the arrival and departure of a fishing boat based on vision. The method comprises the following steps: constructing a target detection algorithm model of the fishing boat and personnel, a tracking algorithm of the fishing boat and personnel, a ship plate detection and identification algorithm model and a classification algorithm model of whether personnel wear life jackets or not; constructing an annual survey fishing boat information database and an forbidden sea time period database; detecting and tracking the fishing boats and personnel in the harbor, identifying whether personnel wear life jackets or not, identifying the ship numbers of the fishing boats, associating an annual review fishing boat information base with a forbidden sea time period database, realizing four types of forbidden sea supervision of non annual review sea, illegal passenger carrying, no wearing life jackets and forbidden sea, and inquiring a preset fishing boat information database through ship plate warning to find out a ship owner for relevant illegal treatment. The invention can provide convenience for the supervisory personnel, thereby further guaranteeing the safety of the fishing boat going in and out of the port.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a method and a system for supervising the arrival and departure of a fishing boat based on vision.
Background
The fishing port is provided with a large number of small three-plate fishing boats, which are not provided with RFID (Radio Frequency Identification ) and AIS (Automatic Identification System, automatic fishing boat identification system), or some fishing boat AIS and other devices are powered off and closed, so that the condition of entering and exiting the fishing boat cannot be monitored by using an AIS positioning system. The monitoring camera is manually monitored or checked in the port on site, so that the labor cost is high, the time period is omitted, and potential safety hazards exist.
The fishing port has the following supervision requirements on the fishing boat: the fishing boat can leave ports after annual inspection is needed to be completed, and the unhatched fishing boat is limited to leave ports; the fishermen on the fishing vessel need to wear life jackets when going out of the sea; the number of the passengers carried by the fishing boat cannot exceed the rated number of the passengers carried by the fishing boat; bad weather and fishing season do not allow departure.
Aiming at the problems, the invention provides a method and a system for supervising the arrival and departure of a fishing boat based on vision.
Disclosure of Invention
In order to overcome the problems in the prior art, the invention provides a method, a system, a device and a computer-readable storage medium for supervising the arrival and departure of a fishing boat based on vision.
The technical scheme for solving the technical problems is as follows:
in a first aspect, the invention provides a vision-based fishing vessel arrival and departure supervision method, which comprises the following steps:
constructing a target detection algorithm model of the fishing boat and personnel, a tracking algorithm of the fishing boat and personnel, a ship plate detection and identification algorithm model and a classification algorithm model of whether personnel wear life jackets or not;
constructing an annual survey fishing boat information database, wherein the annual survey fishing boat information database comprises annual survey fishing boat license information, boat master information and rated fishing boat passenger carrying quantity information; constructing an forbidden sea time period database, wherein the forbidden sea time period database comprises a fishing date and unsuitable sea weather, and the forbidden sea time period is configured according to weather forecast and the fluctuation adjustment condition of the fishing date;
the method comprises the steps of setting a camera to capture videos at preset positions, calling a fishing boat and personnel target detection algorithm model, a fishing boat and personnel tracking algorithm, a boat license detection and identification algorithm model and a personnel wearing life jacket classification algorithm model, detecting and tracking the fishing boat and personnel going out and coming in, identifying personnel wearing life jackets and boat numbers, associating an annual survey fishing boat information base and an illegal sea-going-out time period database, realizing four types of illegal sea-going-out supervision of non annual survey sea, illegal passenger carrying, no wearing life jackets and illegal sea-going-out, searching a ship owner through a boat license warning query, and carrying out relevant illegal treatment.
Further, constructing a fishing boat and personnel target detection algorithm model comprises the following steps: and (3) constructing a fishing boat and personnel data set by capturing and collecting the arrival and departure images of the fishing boat, and training a fishing boat and personnel target detection algorithm model by using a YOLOv8 deep learning target detection algorithm.
Further, the fishing boat and personnel tracking algorithm adopts Deep Sort multi-target tracking algorithm.
Further, constructing a ship board detection and identification algorithm model comprises the following steps: using the fishing boat and personnel data set, and digging the fishing boat area to manufacture a boat plate detection data set; training a ship board detection model by using an R2CNN deep learning detection algorithm;
digging a ship plate in the ship plate detection data set, correcting the ship plate by using a character correction algorithm, and then capturing a picture to manufacture a real ship plate identification data set; synthesizing synthetic ship board identification data sets with different backgrounds and different fonts by taking the ship body as the background and taking the ship boards in the port fishing ship database as the semantics; and manufacturing a ship plate identification data set for training by the ship plate identification data set and the synthesized ship plate identification data set according to the proportion of 1:3, and training a ship plate identification model by using a CRNN deep learning character identification algorithm.
Further, the classifying algorithm model for constructing whether personnel wear the life jacket comprises the following steps: and (3) using a fishing boat and a personnel data set to pick up the personnel area, making whether personnel wear the life jacket data set, and using a ResNet101 deep learning classification recognition algorithm to train whether personnel wear the life jacket classification algorithm model.
Further, detecting and tracking the fishing boats and personnel in the harbor, identifying whether personnel wear life jackets, identifying the ship numbers of the fishing boats, associating an annual review fishing boat information base and an illegal sea-leaving time period database, and realizing four types of illegal sea-leaving supervision of non annual review sea, illegal passenger carrying, non-wearing life jackets and illegal sea-leaving, which comprises the following steps:
calling a trained fishing boat and personnel target detection algorithm model, and identifying fishing boat and personnel targets and areas; and calling a fishing boat and personnel tracking algorithm to track the fishing boat and personnel, determining the arrival and departure directions of the fishing boat, and determining the fishing boat where the personnel are located;
calling a trained ship plate detection and recognition algorithm model, and recognizing the positions of the ship plates of the fishing ship and the ship plates of the fishing ship; comparing the identified ship plate with ship plate data in a pre-constructed fishing ship information database, and if the fishing ship is not in the annual-review fishing ship warehouse, carrying out annual-review sea-going warning;
identifying the result according to the fishing boat and personnel target detection algorithm model, counting the personnel number, comparing the identified fishing boat cards with the rated passenger carrying number of the fishing boat in the fishing boat information base, and carrying out illegal passenger carrying warning if the rated passenger carrying number is exceeded;
according to the identified personnel target area, calling a trained personnel wearing life jacket classification algorithm model to obtain the state of personnel wearing life jackets, and alarming the existence of the personnel without wearing life jackets on the fishing boats with the associated boat cards;
and comparing the time of the arrival and departure of the fishing boat with a database of a preset forbidden sea time period, and carrying out forbidden sea warning on the fishing boat-associated ship plate of the sea in the time period.
Further, the fishing boat and personnel target detection algorithm model, the fishing boat and personnel tracking algorithm, the ship plate detection and identification algorithm model and the personnel wearing life jacket classification algorithm model are stored in a preset database.
In a second aspect, the invention also provides a vision-based monitoring system for the arrival and departure of a fishing boat, which comprises a monitoring station, a data processing module and a monitoring module, wherein:
the monitoring station comprises a camera which is used for tracking and shooting the fishing boat to obtain the image data of the fishing boat;
the data processing module is used for constructing a target detection algorithm model of the fishing boat and personnel, a tracking algorithm of the fishing boat and personnel, a ship plate detection and identification algorithm model and a classification algorithm model of whether personnel wear life jackets or not; constructing an annual survey fishing boat information database, wherein the annual survey fishing boat information database comprises annual survey fishing boat license information, boat master information and rated fishing boat passenger carrying quantity information; constructing a forbidden sea time period database, wherein the forbidden sea time period database comprises a fishing period and bad weather unsuitable for sea going out, and configuring the forbidden sea time period according to weather forecast and the fluctuation adjustment condition of the fishing period;
the supervision module is used for detecting and tracking the incoming and outgoing fishing boats and personnel, identifying whether personnel wear life jackets and identifying the boat numbers of the fishing boats, associating an annual review fishing boat information base with a forbidden sea time period database, realizing four types of forbidden sea supervision of non annual review sea, illegal passenger carrying, non-wearing life jackets and forbidden sea, and inquiring a preset fishing boat information database through a ship plate alarm to find out a boat owner for relevant illegal treatment.
In a third aspect, the present invention also provides a vision-based fishing vessel departure/arrival supervision apparatus, comprising: a processor, a memory, and a program; the program is stored in the memory, and the processor invokes the program stored in the memory to execute a vision-based fishing vessel arrival and departure supervision method according to any one of the embodiments of the first aspect.
In a fourth aspect, the present invention also provides a computer readable storage medium comprising a stored computer program, wherein the computer program, when executed by a processor, controls a device in which the storage medium is located to perform the vision-based fishing vessel ingress and egress supervision method according to any one of the embodiments of the first aspect.
Compared with the prior art, the invention has the following technical effects:
the method comprises the steps of constructing a fishing boat and personnel target detection algorithm model, a fishing boat and personnel tracking algorithm, a ship plate detection and identification algorithm model and a personnel wearing life jacket classification algorithm model; constructing an annual survey fishing boat information database and an forbidden sea time period database; detecting and tracking the fishing boats and personnel in the harbor, identifying whether personnel wear life jackets or not, identifying the ship numbers of the fishing boats, associating an annual review fishing boat information base with a forbidden sea time period database, realizing four types of forbidden sea supervision of non annual review sea, illegal passenger carrying, no wearing life jackets and forbidden sea, and inquiring a preset fishing boat information database through ship plate warning to find out a ship owner for relevant illegal treatment.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of the overall structure of the system of the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the present invention to achieve the preset purpose, the following detailed description is given below of the specific implementation, structure, features and effects of the technical solution according to the present invention with reference to the accompanying drawings and preferred embodiments. The particular features, structures, or characteristics of one or more embodiments may be combined in any suitable manner. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
In another embodiment of the present invention, referring to fig. 1, there is provided a vision-based method for supervising the arrival and departure of a fishing vessel, comprising the steps of:
and 100, constructing a target detection algorithm model of the fishing boat and the personnel, a tracking algorithm model of the fishing boat and the personnel, a ship plate detection and identification algorithm model, and a classification algorithm model of whether the personnel wear life jackets or not, and storing the classification algorithm model into a preset database.
In some embodiments, this step may include the steps of:
step 110: constructing and training a fishing boat and personnel target detection algorithm model;
constructing a target detection algorithm model of the fishing boat and the personnel, wherein the target detection algorithm model of the fishing boat and the personnel adopts a target detection algorithm, the detected fishing boat is used for building a ship plate recognition algorithm model and counting the arrival and departure directions of the fishing boat, and the detected personnel is used for building a classification algorithm model of whether the personnel wear life jackets or not and counting the passenger carrying number of the fishing boat. Specifically, a fishing boat and personnel data set is constructed by capturing and collecting near-ten-thousand fishing boat arrival and departure images, and a model of the fishing boat and personnel target detection algorithm is trained by using a YOLOv8 deep learning target detection algorithm.
Step 120: constructing a fishing boat and personnel tracking algorithm;
and constructing a fishing boat and personnel tracking algorithm, wherein the fishing boat and personnel tracking algorithm is used for determining the direction of the arrival and departure of the fishing boat by tracking the fishing boat, and the fishing boat and personnel tracking algorithm ensures that the detected personnel are on the fishing boat by tracking personnel, and prevents misjudgment of personnel on the boat when a plurality of ships are on video pictures. Specifically, the fishing boat and personnel tracking algorithm adopts a Deep Sort multi-target tracking algorithm.
Step 130: constructing and training a ship plate detection and recognition algorithm model;
using the fishing boat and personnel data set constructed in the step 110, and digging the fishing boat area to manufacture a boat plate detection data set so as to reduce background interference; training a ship board detection model by using an R2CNN deep learning detection algorithm;
and (3) picking up the ship plate of the ship plate detection data set, correcting the ship plate by using a character correction algorithm, and then capturing a picture to manufacture a real ship plate identification data set. In addition, the ship body is taken as a background, and the ship cards in the port fishing ship database are taken as semantics, so that the synthesized ship card identification data sets with different backgrounds and different fonts are synthesized. And manufacturing a ship plate identification data set for training by the ship plate identification data set and the synthesized ship plate identification data set according to the proportion of 1:3, training a ship plate identification model by using a CRNN deep learning character identification algorithm, and storing the ship plate identification model in a preset database. And calling a trained ship plate detection and recognition algorithm model to recognize the fishing plate of the target fishing ship.
Step 140: constructing and training a life jacket classification algorithm model of whether personnel wear the life jacket;
and constructing a life jacket wearing or not by the personnel, wherein the life jacket wearing or not by the personnel is judged by adopting an image classification algorithm. Specifically, the fishing boat and personnel data set constructed in the step 110 are used for picking personnel areas, whether personnel wear life jacket data sets or not is made, a ResNet101 deep learning classification recognition algorithm is used for training personnel whether to wear life jacket classification algorithm models or not, and the life jacket classification algorithm models are stored in a preset database. And calling a trained personnel wearing life jacket classification algorithm model to identify whether a target personnel wears life jackets.
Step 200, constructing an annual survey fishing boat information database, wherein the fishing boat information database comprises annual survey fishing boat license information, boat master information and rated fishing boat passenger carrying quantity information; constructing an forbidden sea time period database, wherein the forbidden sea time period database comprises a fishing period and unsuitable sea weather, and the forbidden sea time period can be configured according to weather forecast and the fluctuation adjustment condition of the fishing period.
Step 210: constructing an annual survey fishing boat information database;
the fishing boat information base comprises annual survey fishing boat plate information, boat main information, rated fishing boat passenger carrying quantity information and the like, and is used for comparing the target fishing boat, the target fishing boat plate and the annual survey fishing boat information base and judging whether the sea target fishing boat is annual survey or not.
Step 220: constructing a configurable forbidden sea time period database;
constructing a forbidden sea time period database which contains bad weather unsuitable for sea going out, such as a fishing break, strong wind, strong fog and the like. The database time period configuration can be configured according to severe weather forecast, holiday change adjustment and other conditions. The method is used for comparing the sea-going time of the target fishing boat with the forbidden sea-going time period database and judging whether the sea target fishing boat is forbidden to go out of the sea.
Step 300, setting a camera to capture videos at preset positions, calling a target detection algorithm model of fishing boats and personnel, a tracking algorithm of the fishing boats and the personnel, a ship brand detection and identification algorithm model and a personnel wearing life jacket classification algorithm model, detecting and tracking the fishing boats and the personnel on the port, identifying whether the personnel wearing life jackets and the number of the fishing boats, associating an annual survey fishing boat information base and an illegal sea-going-out time period database, realizing four types of illegal sea-going-out supervision of the non annual survey sea, illegal passenger carrying, no wearing life jackets and illegal sea-going-out supervision, searching a ship owner through a ship brand alarm query preset annual survey fishing boat information database, and carrying out relevant illegal treatment.
In some embodiments, this step may include the steps of:
step 310: calling a trained fishing boat and personnel target detection algorithm model, and identifying fishing boat and personnel targets and areas; and calling a fishing boat and personnel tracking algorithm to track the fishing boat and personnel, determining the arrival and departure directions of the fishing boat, and determining the fishing boat where the personnel are located.
Step 320: calling a trained ship plate detection and recognition algorithm model, and recognizing the positions of the ship plates of the fishing ship and the ship plates of the fishing ship; comparing the identified ship plate with ship plate data of a pre-constructed annual survey fishing ship information database, if the fishing ship is not in the annual survey fishing ship database, carrying out an annual survey sea warning, and storing video information 15 seconds before and after the violation;
step 330: according to the personnel targets identified in the step 310, the personnel number is calculated, and according to the fishing boat cards identified in the step 320, the personnel number is compared with the rated passenger carrying number of the fishing boat in the fishing boat information base, if the personnel number exceeds the rated passenger carrying number, the warning of illegal passenger carrying is carried out, and the video information 15 seconds before and after the violation is stored;
step 340: according to the personnel target area identified in the step 310, calling a trained personnel wearing life jacket classification algorithm model to obtain the state that personnel wearing life jackets, alarming the existence of the fishing boat with the personnel not wearing life jackets by the fishing boat cards identified in the association step 320, and storing video information 15 seconds before and after the violation;
step 350: and comparing the time of arrival and departure of the fishing boat with a preset forbidden sea time database, carrying out forbidden sea warning on the fishing boat cards identified in the step 320 of correlation of the fishing boat for sea departure in the time period, and storing video information 15 seconds before and after the violation.
Through the implementation of the provided monitoring system for the arrival and departure of the fishing boat based on vision, convenience can be provided for monitoring staff, so that the arrival and departure safety of the fishing boat is further ensured.
In another embodiment of the present invention, a vision-based fishing vessel ingress and egress monitoring system is provided, comprising at least one monitoring station, at least one data processing module, and at least one monitoring module, wherein:
the monitoring station comprises a camera which is used for tracking and shooting the fishing boat to acquire the fishing boat image data.
The data processing module is used for constructing a target detection algorithm model of the fishing boat and personnel, a tracking algorithm of the fishing boat and personnel, a ship plate detection and identification algorithm model and a classification algorithm model of whether personnel wear life jackets or not; constructing an annual survey fishing boat information database, wherein the annual survey fishing boat information database comprises annual survey fishing boat license information, boat master information and rated fishing boat passenger carrying quantity information; constructing an forbidden sea time period database, wherein the forbidden sea time period database comprises a fishing period and bad weather unsuitable for sea going out;
the supervision module is used for detecting and tracking the incoming and outgoing fishing boats and personnel, identifying whether personnel wear life jackets and identifying the boat numbers of the fishing boats, associating an annual review fishing boat information base with a forbidden sea time period database, realizing four types of forbidden sea supervision of non annual review sea, illegal passenger carrying, non-wearing life jackets and forbidden sea, and inquiring a preset fishing boat information database through a ship plate alarm to find out a boat owner for relevant illegal treatment.
In this embodiment, the data processing module includes: the device comprises a fishing boat and personnel detection module, a fishing boat and personnel tracking module, a ship board detection and identification module, a personnel wearing life jacket classification module, a annual inspection fishing boat discrimination module and a sea time discrimination module.
And the fishing boat and personnel detection module is used for constructing and training a fishing boat and personnel target detection algorithm model and detecting target fishing boats and target personnel. The target detection algorithm model of the fishing boat and personnel adopts a target detection algorithm, the detected fishing boat is used for building a ship plate recognition algorithm model and counting the arrival and departure directions of the fishing boat, and the detected personnel is used for building a life jacket wearing classification algorithm model and counting the number of passengers carried by the fishing boat. Specifically, near-ten-thousand fishing boat arrival and departure images are collected through snapshot to construct a fishing boat and personnel data set, a YOLOv8 deep learning target detection algorithm is used for training a fishing boat and personnel target detection algorithm model, the trained fishing boat and personnel target detection algorithm model is called, and target fishing boat and target personnel in the collected fishing boat image data are identified.
And a fishing boat and personnel tracking module for constructing a fishing boat and personnel tracking algorithm. The fishing boat and personnel tracking algorithm is used for determining the direction of the arrival and departure of the fishing boat by tracking the fishing boat, and the fishing boat and personnel tracking algorithm ensures that the detected personnel are on the fishing boat by tracking personnel, and when a plurality of ships appear in a video picture, misjudgment of the personnel on the ship is prevented. Specifically, the fishing boat and personnel tracking algorithm adopts a Deep Sort multi-target tracking algorithm.
And the ship plate detection and identification module is used for constructing and training a ship plate detection and identification algorithm model for identifying the fishing plate of the target fishing boat. Using a fishing boat and personnel data set, and digging a fishing boat area to manufacture a ship plate detection data set so as to reduce background interference; training a ship board detection model by using an R2CNN deep learning detection algorithm; and (3) picking up the ship plate of the ship plate detection data set, correcting the ship plate by using a character correction algorithm, and then capturing a picture to manufacture a real ship plate identification data set. In addition, by taking the ship body as a background and taking the ship plate in the port fishing ship database as the semantic, synthesizing synthetic ship plate identification data sets with different backgrounds and different fonts; and manufacturing a ship plate identification data set for training by the ship plate identification data set and the synthesized ship plate identification data set according to the proportion of 1:3, training a ship plate identification model by using a CRNN deep learning character identification algorithm, and storing the ship plate identification model in a preset database. And calling a trained ship plate detection and recognition algorithm model to recognize the fishing plate of the target fishing ship.
And the personnel wearing life jacket classifying module is used for constructing and training a life jacket classifying algorithm model for identifying whether target personnel wear life jackets. The personnel wearing life jacket classifying algorithm module adopts an image classifying algorithm to judge whether personnel wearing life jackets. Specifically, a fishing boat and a personnel data set are used for picking personnel areas, whether personnel wear life jacket data sets or not is made, a ResNet101 deep learning classification recognition algorithm is used, whether personnel wear life jacket classification algorithm models are trained, and the life jacket classification algorithm models are stored in a preset database. And calling a trained personnel wearing life jacket classification algorithm model to identify whether a target personnel wears life jackets.
The annual survey fishing boat judging module is used for constructing an annual survey fishing boat information database, comparing the identified fishing boat cards with data in the annual survey fishing boat information database, and judging whether the sea-going target fishing boat is annual survey or not, wherein the fishing boat information database comprises annual survey fishing boat card information, boat host information, rated fishing boat passenger carrying quantity information and the like.
The sea-going-out time judging module is used for constructing a forbidden sea-going-out time period database and judging whether sea-going-out is possible according to severe weather and fishing holiday factors. Specifically, a forbidden sea time period database is constructed, wherein the forbidden sea time period database comprises bad weather unsuitable for sea going out, such as fishing season, strong wind, strong fog and the like. The time period configuration of the database can be configured according to severe weather forecast, holiday change adjustment and other conditions.
In an embodiment of the present invention, there is also provided a vision-based fishing vessel arrival and departure supervision apparatus including: a processor, a memory, and a program; the program is stored in the memory, and the processor calls the program stored in the memory to execute the vision-based fishing boat arrival and departure supervision method.
In the implementation of the vision-based fishing vessel departure/arrival supervision device, the memory and the processor are directly or indirectly electrically connected to realize data transmission or interaction. For example, the elements may be electrically connected to each other via one or more communication buses or signal lines, such as through a bus connection. The memory stores computer-executable instructions for implementing the data access control method, including at least one software functional module that may be stored in the memory in the form of software or firmware, and the processor executes the software programs and modules stored in the memory to perform various functional applications and data processing.
The Memory may be, but is not limited to, random access Memory (Random Access Memory; RAM), read Only Memory (ROM), programmable Read Only Memory (Programmable Read-Only Memory; PROM), erasable Read Only Memory (Eraseable)
Programmable Read-Only Memory, abbreviated: EPROM), electrically erasable read-Only Memory (Electric Erasable Programmable Read-Only Memory, abbreviation: EEPROM), and the like. The memory is used for storing a program, and the processor executes the program after receiving the execution instruction.
The processor may be an integrated circuit chip with signal processing capabilities. The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), and the like. The disclosed methods, steps, and logic blocks in the embodiments of the present invention may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In an embodiment of the present invention, there is further provided a computer readable storage medium, including a stored computer program, where the computer program, when executed by a processor, controls a device in which the storage medium is located to perform the above-mentioned method for supervising the arrival and departure of a fishing vessel based on vision.
It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the invention may take the form of an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the invention may take the form of a computer program product on one or more computer-usable storage media having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart.
The above describes in detail the application of the vision-based fishing boat entry and exit supervision method, the vision-based fishing boat entry and exit supervision system, the vision-based fishing boat entry and exit supervision device and a computer-readable storage medium, and specific examples are applied to the present invention to illustrate the principles and embodiments of the present invention, and the above description of the examples is only used to help understand the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.
The above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (10)
1. The method for supervising the arrival and departure of the fishing boat based on vision is characterized by comprising the following steps:
constructing a target detection algorithm model of the fishing boat and personnel, a tracking algorithm of the fishing boat and personnel, a ship plate detection and identification algorithm model and a classification algorithm model of whether personnel wear life jackets or not;
constructing an annual survey fishing boat information database, wherein the annual survey fishing boat information database comprises annual survey fishing boat license information, boat master information and rated fishing boat passenger carrying quantity information; constructing an forbidden sea time period database, wherein the forbidden sea time period database comprises a fishing date and unsuitable sea weather, and the forbidden sea time period is configured according to weather forecast and the fluctuation adjustment condition of the fishing date;
setting a camera to capture videos at preset positions, calling a fishing boat and personnel tracking algorithm, a trained fishing boat and personnel target detection algorithm model, a trained ship plate detection and identification algorithm model and a trained personnel wearing life jacket classification algorithm model, detecting and tracking the fishing boat and personnel going in and out, identifying personnel wearing life jackets and fishing boat plates, associating an annual fishing boat information base and an illegal sea-going-out time period database, realizing four types of illegal sea-going-out supervision of the non annual sea, illegal passenger carrying, non-wearing life jackets and illegal sea-going-out supervision, searching a ship owner through a ship plate alarm query preset annual fishing boat information database, and carrying out relevant illegal treatment.
2. The vision-based fishing vessel arrival and departure supervision method according to claim 1, wherein constructing a fishing vessel and personnel objective detection algorithm model comprises: and (3) constructing a fishing boat and personnel data set by capturing and collecting the arrival and departure images of the fishing boat, and training a fishing boat and personnel target detection algorithm model by using a YOLOv8 deep learning target detection algorithm.
3. The vision-based fishing vessel arrival and departure supervision method according to claim 1, wherein the fishing vessel and personnel tracking algorithm adopts Deep Sort multi-objective tracking algorithm.
4. The vision-based fishing vessel arrival and departure supervision method according to claim 2, wherein constructing a ship plate detection and recognition algorithm model comprises: using the fishing boat and personnel data set, and digging the fishing boat area to manufacture a boat plate detection data set; training a ship board detection model by using an R2CNN deep learning detection algorithm;
digging a ship plate in the ship plate detection data set, correcting the ship plate by using a character correction algorithm, and then capturing a picture to manufacture a real ship plate identification data set; synthesizing synthetic ship board identification data sets with different backgrounds and different fonts by taking the ship body as the background and taking the ship boards in the port fishing ship database as the semantics; and manufacturing a ship plate identification data set for training by the ship plate identification data set and the synthesized ship plate identification data set according to the proportion of 1:3, and training a ship plate identification model by using a CRNN deep learning character identification algorithm.
5. The vision-based fishing vessel ingress and egress monitoring method of claim 2 wherein constructing a model of a life jacket classification algorithm comprises: and (3) using a fishing boat and a personnel data set to pick up the personnel area, making whether personnel wear the life jacket data set, and using a ResNet101 deep learning classification recognition algorithm to train whether personnel wear the life jacket classification algorithm model.
6. The method for supervising the arrival and departure of the fishing vessel based on the vision according to claim 1, wherein the method for supervising the arrival and departure of the fishing vessel based on the vision comprises the steps of detecting and tracking the fishing vessel and personnel in the arrival and departure, identifying whether personnel wear life jackets and ship cards of the fishing vessel, associating an annual inspected fishing vessel information base and an forbidden sea time period database, and realizing four types of forbidden sea supervision of non annual inspected sea, illegal passenger carrying, non-wearing life jackets and forbidden sea, and the method specifically comprises the following steps:
calling a trained fishing boat and personnel target detection algorithm model, and identifying fishing boat and personnel targets and areas; tracking the fishing boat and personnel by using a fishing boat and personnel tracking algorithm, determining the arrival and departure directions of the fishing boat, and determining the fishing boat where the personnel are located;
calling a trained ship plate detection and recognition algorithm model, and recognizing the positions of the ship plates of the fishing ship and the ship plates of the fishing ship; comparing the identified ship plate with ship plate data in a pre-constructed fishing ship information database, and if the fishing ship is not in the annual-review fishing ship warehouse, carrying out annual-review sea-going warning;
identifying the result according to the fishing boat and personnel target detection algorithm model, counting the personnel number, comparing the identified fishing boat cards with the rated passenger carrying number of the fishing boat in the fishing boat information base, and carrying out illegal passenger carrying warning if the rated passenger carrying number is exceeded;
according to the identified personnel target area, calling a trained personnel wearing life jacket classification algorithm model to obtain the state of personnel wearing life jackets, and alarming the existence of the personnel without wearing life jackets on the fishing boats with the associated boat cards;
and comparing the time of the arrival and departure of the fishing boat with a database of a preset forbidden sea time period, and carrying out forbidden sea warning on the fishing boat-associated ship plate of the sea in the time period.
7. The vision-based fishing vessel arrival and departure supervision method according to claim 1, wherein the fishing vessel and personnel target detection algorithm model, the fishing vessel and personnel tracking algorithm, the ship plate detection and identification algorithm model and the personnel wearing life jacket classification algorithm model are stored in a preset database.
8. The utility model provides a fishing boat business turn over port supervisory systems based on vision which characterized in that includes monitor site, data processing module and supervision module, wherein:
the monitoring station comprises a camera which is used for tracking and shooting the fishing boat to obtain the image data of the fishing boat;
the data processing module is used for constructing a target detection algorithm model of the fishing boat and personnel, a tracking algorithm of the fishing boat and personnel, a ship plate detection and identification algorithm model and a classification algorithm model of whether personnel wear life jackets or not; constructing an annual survey fishing boat information database, wherein the annual survey fishing boat information database comprises annual survey fishing boat license information, boat master information and rated fishing boat passenger carrying quantity information; constructing a forbidden sea time period database, wherein the forbidden sea time period database comprises a fishing period and bad weather unsuitable for sea going out, and configuring the forbidden sea time period according to weather forecast and the fluctuation adjustment condition of the fishing period;
the supervision module is used for detecting and tracking the incoming and outgoing fishing boats and personnel, identifying whether personnel wear life jackets and identifying the boat numbers of the fishing boats, associating an annual review fishing boat information base with a forbidden sea time period database, realizing four types of forbidden sea supervision of non annual review sea, illegal passenger carrying, non-wearing life jackets and forbidden sea, and inquiring a preset fishing boat information database through a ship plate alarm to find out a boat owner for relevant illegal treatment.
9. The utility model provides a fishing boat entering and leaving port supervision device based on vision which characterized in that it includes: a processor, a memory, and a program; the program is stored in the memory, and the processor invokes the memory-stored program to perform the method of any one of claims 1-7.
10. A computer readable storage medium, characterized in that the computer readable storage medium comprises a stored computer program, wherein the computer program, when run by a processor, controls a device in which the storage medium is located to perform the method of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311329900.6A CN117115765B (en) | 2023-10-16 | 2023-10-16 | Fishing boat arrival and departure supervision method and system based on vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311329900.6A CN117115765B (en) | 2023-10-16 | 2023-10-16 | Fishing boat arrival and departure supervision method and system based on vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117115765A true CN117115765A (en) | 2023-11-24 |
| CN117115765B CN117115765B (en) | 2024-01-09 |
Family
ID=88809317
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311329900.6A Active CN117115765B (en) | 2023-10-16 | 2023-10-16 | Fishing boat arrival and departure supervision method and system based on vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117115765B (en) |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020140577A1 (en) * | 2001-01-26 | 2002-10-03 | Kavner Douglas M. | System and method for reading license plates |
| KR20160089694A (en) * | 2015-01-20 | 2016-07-28 | 재단법인 중소조선연구원 | Safety work management system for fishing ship and the method thereof |
| CN110046547A (en) * | 2019-03-06 | 2019-07-23 | 深圳市麦谷科技有限公司 | Report method, system, computer equipment and storage medium violating the regulations |
| CN110110964A (en) * | 2019-04-04 | 2019-08-09 | 深圳市云恩科技有限公司 | A kind of ship and ferry supervisory systems based on deep learning |
| CN110288006A (en) * | 2019-06-03 | 2019-09-27 | 上海眼控科技股份有限公司 | A kind of license plate number automatic Verification method and system |
| KR102137380B1 (en) * | 2019-11-05 | 2020-07-24 | 에이원커뮤니케이션즈코리아(주) | Management system of smart harbor |
| CN112380914A (en) * | 2020-10-21 | 2021-02-19 | 浙江工业大学 | Fishing boat safety monitoring method based on deep learning |
| KR20220065384A (en) * | 2020-11-13 | 2022-05-20 | 한국해양대학교 산학협력단 | Ship entry/departure monitoring system using real-time port images and ship identification device signals |
| CN115100477A (en) * | 2022-07-06 | 2022-09-23 | 海南大学 | Intelligent fishing port supervision system and method based on digital twin |
| CN115700817A (en) * | 2022-10-20 | 2023-02-07 | 华南师范大学 | Fishery safety monitoring method and metas based on artificial intelligence and digital twin |
| CN116206489A (en) * | 2022-12-16 | 2023-06-02 | 南京莱斯网信技术研究院有限公司 | Ship traceability and health index monitoring method based on dynamic event |
-
2023
- 2023-10-16 CN CN202311329900.6A patent/CN117115765B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20020140577A1 (en) * | 2001-01-26 | 2002-10-03 | Kavner Douglas M. | System and method for reading license plates |
| KR20160089694A (en) * | 2015-01-20 | 2016-07-28 | 재단법인 중소조선연구원 | Safety work management system for fishing ship and the method thereof |
| CN110046547A (en) * | 2019-03-06 | 2019-07-23 | 深圳市麦谷科技有限公司 | Report method, system, computer equipment and storage medium violating the regulations |
| CN110110964A (en) * | 2019-04-04 | 2019-08-09 | 深圳市云恩科技有限公司 | A kind of ship and ferry supervisory systems based on deep learning |
| CN110288006A (en) * | 2019-06-03 | 2019-09-27 | 上海眼控科技股份有限公司 | A kind of license plate number automatic Verification method and system |
| KR102137380B1 (en) * | 2019-11-05 | 2020-07-24 | 에이원커뮤니케이션즈코리아(주) | Management system of smart harbor |
| CN112380914A (en) * | 2020-10-21 | 2021-02-19 | 浙江工业大学 | Fishing boat safety monitoring method based on deep learning |
| KR20220065384A (en) * | 2020-11-13 | 2022-05-20 | 한국해양대학교 산학협력단 | Ship entry/departure monitoring system using real-time port images and ship identification device signals |
| CN115100477A (en) * | 2022-07-06 | 2022-09-23 | 海南大学 | Intelligent fishing port supervision system and method based on digital twin |
| CN115700817A (en) * | 2022-10-20 | 2023-02-07 | 华南师范大学 | Fishery safety monitoring method and metas based on artificial intelligence and digital twin |
| CN116206489A (en) * | 2022-12-16 | 2023-06-02 | 南京莱斯网信技术研究院有限公司 | Ship traceability and health index monitoring method based on dynamic event |
Non-Patent Citations (3)
| Title |
|---|
| A. PRAYUDI等: ""Surveillance System for Illegal Fishing Prevention on UAV Imagery Using Computer Vision"", 《2020 INTERNATIONAL ELECTRONICS SYMPOSIUM (IES)》 * |
| 冯喜惠等: ""基于深度学习的图像识别技术在渔船监管中的应用"", 《珠江水运》, vol. 2022, no. 23 * |
| 周全;倪军;顾方;: "信息化在渔港渔船监管上的应用研究――以上海市渔港渔船监管系统为例", 中国管理信息化, no. 12 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117115765B (en) | 2024-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101824446B1 (en) | A reinforcement learning based vehicle number recognition method for CCTV | |
| CN108806334A (en) | A kind of intelligent ship personal identification method based on image | |
| CN110070155B (en) | Comprehensive behavior recognition method and system for prisoner based on wearable equipment | |
| CN111935446B (en) | Ship monitoring method, device, monitoring system and equipment based on block chain | |
| CN113628404A (en) | Method and device for reducing invalid alarm | |
| Bloisi et al. | Camera based target recognition for maritime awareness | |
| CN116453276A (en) | Marine wind power electronic fence monitoring and early warning method and system | |
| CN113052107A (en) | Method for detecting wearing condition of safety helmet, computer equipment and storage medium | |
| CN111797726A (en) | Flame detection method and device, electronic equipment and storage medium | |
| CN115410112A (en) | Elevator abnormal stay recognition method and device and storage medium | |
| CN209312052U (en) | A kind of carry-on police system of recognition of face | |
| CN111461655A (en) | Personnel management system | |
| CN117115765B (en) | Fishing boat arrival and departure supervision method and system based on vision | |
| CN113823054A (en) | Comprehensive security management system, method and device for converter station and computer equipment | |
| CN102595106A (en) | Device and method for harbor frontier inspection intelligent monitoring inspection | |
| CN117237935A (en) | Method and device for identifying space object, electronic equipment and storage medium | |
| CN115019463B (en) | Water area supervision system based on artificial intelligence technology | |
| CN116052356A (en) | Intelligent building site monitor platform | |
| CN116012939A (en) | Method and device for determining illegal behaviors, storage medium and electronic device | |
| CN118135752B (en) | Intelligent early warning all-in-one based on video monitoring system | |
| CN115376275B (en) | Construction safety warning method and system based on image processing | |
| CN118609171B (en) | Pet biting behavior identification method, device, equipment and medium based on vision | |
| CN117522351B (en) | Locking and listing method and system based on data analysis | |
| Yuan et al. | A constructing vehicle intrusion detection algorithm based on BOW presentation model | |
| CN116092272A (en) | Intelligent warning system for three-dimensional sea defense |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |