CN111263111B - Long line fishing information extraction system based on surveillance video - Google Patents
Long line fishing information extraction system based on surveillance video Download PDFInfo
- Publication number
- CN111263111B CN111263111B CN202010050955.3A CN202010050955A CN111263111B CN 111263111 B CN111263111 B CN 111263111B CN 202010050955 A CN202010050955 A CN 202010050955A CN 111263111 B CN111263111 B CN 111263111B
- Authority
- CN
- China
- Prior art keywords
- fishing
- information
- video
- character
- picture
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to a long line fishing information extraction system based on a monitoring video, which comprises: the system comprises a video acquisition device and a monitoring host; the video acquisition device is used for acquiring longline fishing video image information and fish capture video image information; the monitoring host comprises a character extraction module, a longline fishing main line receiving and releasing information extraction module and a fish catching information extraction module; the character extraction module is used for intercepting character image areas in the longline fishing video image information and the fish capturing video image information, and extracting character information from the character image areas to obtain information of the position and the navigation state of the fishing boat; the long line fishing main line receiving and releasing information extraction module is used for extracting the length of a main line in the long line fishing video image information; the fish catching information extraction module is used for extracting the quantity information of the caught fishes in the longline fishing video image information. The method can extract information such as fishing operation, time, position, navigation state and the like in the video.
Description
Technical Field
The invention relates to the technical field of ocean fishing boat fishing monitoring, in particular to a long line fishing information extraction system based on a monitoring video.
Background
In recent years, video monitoring technology is rapidly developed and is applied to various fields such as automatic production, unmanned driving, automatic vending and the like. The video monitoring technology extracts information in the video for production and life by means of strong data processing capacity of a computer, and effectively promotes social informatization. In the process of fishing by the ocean fishing boat, in order to ensure the safe production of the fishing boat, record the operation of the fishing boat and monitor the state of the fishing boat, video monitors are arranged on the front deck, the rear deck, the port side, the starboard side and the like of the fishing boat. The video monitoring operation process is used for assisting fishery safe production and management, so that convenience is brought to fishery enterprises, and fishing boat management is ordered, reasonable and efficient. The monitoring system is connected with the GPS terminal, and information such as time, position, course, navigational speed and the like is embedded in the video image. Computer image processing technology develops rapidly, and image-based information extraction technology is widely applied. EMS (electronic Monitoring System) in ocean fishing is called "electronic observer". Is expected to replace the current observers. Ocean fishing boats have various operation types, the monitoring and mounting positions of the fishing boats have large difference, and no unified standard exists at present.
Disclosure of Invention
The invention aims to provide a long line fishing information extraction system based on a monitoring video, which can extract information such as fishing operation, time, position, navigation state and the like in the video.
The technical scheme adopted by the invention for solving the technical problems is as follows: the utility model provides a longline fishing information extraction system based on surveillance video, includes: the system comprises a video acquisition device and a monitoring host; the video acquisition device is used for acquiring longline fishing video image information and fish capture video image information; the monitoring host comprises a character extraction module, a longline fishing main line receiving and releasing information extraction module and a fish catching information extraction module; the character extraction module is used for intercepting character image areas in the longline fishing video image information and the fish capturing video image information, and extracting character information from the character image areas to obtain information of the position and the navigation state of the fishing boat; the long line fishing main line receiving and releasing information extraction module is used for extracting the length of a main line in the long line fishing video image information; the fish catching information extraction module is used for extracting the quantity information of the caught fishes in the longline fishing video image information.
The video acquisition device comprises a first camera, a second camera, a third camera and a fourth camera; the first camera is positioned on the starboard of the fishing boat and used for acquiring image information of long line fishing branch line recovery, fish capture and fish unloading of the fishing boat; the second camera is positioned on the port of the fishing boat and used for acquiring the image information of the fish in and out of the bin; the third camera is positioned at the top of the front deck and is used for acquiring image information of processing of the fish on the deck; and the fourth camera is positioned on the rear deck and used for acquiring the winding and unwinding conditions of the long-line fishing line.
The character extraction module comprises a character and picture intercepting unit, a character and picture processing unit and a picture and character extraction unit; the character picture intercepting unit intercepts time pictures and navigation information pictures in the video image information of longline fishing and the video image information of fish capturing according to a preset position; the character picture processing unit carries out graying processing on the intercepted time picture and navigation information picture by using an integer algorithm and then realizes binarization processing through threshold value conversion; and the picture character extraction unit identifies the picture after binarization processing through an identification tool and extracts time, longitude and latitude, course and speed.
The character extraction module further comprises a character recognition training unit, and the character recognition training unit takes each frame in the video image information of longline fishing and the video image information of fish capture as a training sample to train a user-defined word stock.
The long line fishing main line receiving and releasing information extraction module carries out the following processing on the long line fishing video image information: firstly, graying the longline video image information, then smoothing the brightness of pixels by using a low-pass filter, then performing Gaussian filtering, and performing difference processing on the grayed picture and the Gaussian filtered picture to obtain a final picture, wherein the point of the intersection of a reel axis line and a reel disc in the final picture is marked as O, the point on the outer ring of the reel disc is marked as C, the pixel number of the reel disc OC when the reel axis is not provided with the line is marked as B, the actual length is B, the length of the main line to be released is obtained by calculating L- (K/D) x (π x (M- (N × B/B) x 2) + π N) x ((N × B/B-N) D/2), wherein L is the total length of the main line, K is the length of the reel, D is the diameter of the main line, M is the diameter of the external disk of the reel shaft, N is the diameter of the reel shaft without the main line, and N is the number of pixels of the reel OC in the process of winding and unwinding the main line.
The fish catching information extraction module marks labels on pictures of fish catching and floating ball recovery, the pictures are trained through a deep learning model, the number of caught fishes, the paying-off length and the catching efficiency are counted through the operation process of once paying off and collecting the net lines according to the position and the navigation state information of a fishing boat when the fishes are caught and the length of the main line when the main line drawing-off information extraction module extracts the floating balls for rope catching, wherein the first floating ball for rope catching operation recovery is marked as N1, the last floating ball is marked as Nk, the middle floating ball is Ni, the position when the main line paying-off is finished is (lon, lat), the length of the middle floating ball Ni paying-off line is li, the direction angle is alpha, the longitude of the floating ball Ni is Longi ═ lon + Li ÷ sin α ÷ [ R ÷ cos (lat) x 2 π ÷ 360 ÷ latitude ], the lati ═ lat + li × cos α ÷ R ]/, the floating ball Ni is 360 ÷ Lat ÷ lat ÷, r is the radius of the earth.
Advantageous effects
Due to the adoption of the technical scheme, compared with the prior art, the invention has the following advantages and positive effects: according to the invention, in the fishing production process of the longline fishing boat, fishing information is extracted based on the monitoring video according to the monitoring video acquired by the monitoring camera, the process of longline fishing operation can be acquired, the number of the tunas to be fished and the arrangement condition of the floating balls can be counted, the fishery production and management can be assisted, convenience is brought to fishery enterprises, and the management of the fishing boat is orderly and efficiently realized. Information such as time, position, navigation state, unhooking start-stop time, tuna catching time and the like extracted from the video can be inquired by fishery enterprises, fishing conditions can be counted, and aquatic product catching can be traced; the consumer can inquire and know the catching time and position of the purchased tuna and catch screenshots and images; the fishery management organization can monitor whether illegal fishing exists or not.
Drawings
FIG. 1 is a schematic structural view of the present invention;
FIG. 2 is a schematic diagram of the position of the text message embedded in the picture according to the embodiment of the present invention;
FIG. 3 is a schematic diagram of a captured text image after graying processing and binarization processing in an embodiment of the invention;
FIG. 4 is a schematic diagram of manually labeling characters in an embodiment of the present invention;
FIG. 5 is a schematic view of the position and navigation state of the fishing vessel extracted in the embodiment of the present invention;
FIG. 6 is a diagram of surveillance video acquired by Camera No. 4 in an embodiment of the present invention;
FIG. 7 is a schematic diagram of a final picture obtained by the retrieval module of the main line collecting and releasing information of longline fishing according to the embodiment of the present invention;
FIG. 8 is a graph of tuna capture surveillance video in accordance with an embodiment of the invention;
FIG. 9 is a video image of a float ball recovery monitor according to an embodiment of the present invention;
FIG. 10 is a graph of monitor video collected by a branch line in an embodiment of the invention;
fig. 11 is a video of tuna unloading and processing monitoring in an embodiment of the invention.
Detailed Description
The invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Further, it should be understood that various changes or modifications of the present invention may be made by those skilled in the art after reading the teaching of the present invention, and such equivalents may fall within the scope of the present invention as defined in the appended claims.
The embodiment of the invention relates to a long line fishing information extraction system based on a monitoring video, as shown in figure 1, comprising: the system comprises a video acquisition device and a monitoring host; the video acquisition device is used for acquiring longline fishing video image information and fish capture video image information; the monitoring host comprises a character extraction module, a longline fishing main line receiving and releasing information extraction module and a fish catching information extraction module; the character extraction module is used for extracting character information in the video image information of longline fishing and the video image information of fish capture so as to obtain information of the position and navigation state of the fishing boat; the long line fishing main line receiving and releasing information extraction module is used for extracting the length of a main line in the long line fishing video image information; the fish catching information extraction module is used for extracting the quantity information of the caught fishes in the longline fishing video image information.
The present invention will be further explained by applying the present embodiment to a tuna longline fishing boat.
A monitoring system is installed on a tuna longline fishing boat, a monitoring host is connected with a camera, a GPS and other devices, information such as fishing boat operation images, a real-time GPS, personnel activities, running conditions of the fishing boat and the like is collected, omnibearing monitoring on the boat is realized, and various information data are stored locally. 4 cameras are installed on the fishing boat, and a shipborne display screen displays video images in real time.
The fishing boat monitoring system realizes the functions of acquisition, transmission, data storage and management. The monitoring host adopts a wall-mounted installation mode, the front-end camera is directly connected to the network through a network cable, and video signals and control signals are transmitted through IP packets. The monitoring host machine realizes the centralized management of the front-end equipment and the storage of video streaming media, and the system functions comprise the functions of system configuration, real-time monitoring, video query, video playback, real-time control, alarm management, image capture, record capture and the like. The remote terminal can also obtain monitoring information through satellite communication.
The monitoring positions of the tuna longline fishing boat mainly comprise a port and starboard, a front deck and a rear deck (see figure 1). The No. 1 camera is positioned on the starboard of the fishing boat and is used for shooting the recovery of a longline fishing branch line, the capture of tuna and the unloading of fish catches by the fishing boat; the No. 2 camera is positioned on the port of the fishing boat and is used for shooting the tuna gains to enter and exit the bin; the No. 3 camera is positioned at the top of the front deck, the lens direction is backward, and the simple processing of the tuna on the shooting deck is carried out; the No. 4 camera is positioned on the rear deck, the lens direction is forward, and the condition of taking and releasing the long-line fishing line is shot.
The monitoring videos are automatically stored, a folder is dynamically generated every day, each video monitoring is stored once every hour, the file name is stored, the name of the file is ship name-camera number-year-month-day-hour-minute-longitude-latitude', time (UTC), ship name, longitude, latitude, course and navigation speed information is arranged on a video image, and in order to improve the accuracy of information extraction, the information adopts white fonts and black frames. In this embodiment, the video output has a picture resolution of 704 × 576, a horizontal and vertical resolution of 96dpi, and a bit depth of 24. In order to improve the extraction efficiency of the text information, the position of the text information is fixed in the whole frame, the position expression format is [ line beginning: line end, line beginning: line end ], the position of the time picture in the frame is [19:51,448:676] (mark X in FIG. 2), the position of the navigation information picture (ship name, longitude and latitude, course, speed) in the frame is [451:572,565:696] (mark Y in FIG. 2).
The character extraction module in the embodiment comprises a character picture intercepting unit, a character picture processing unit and a picture character extraction unit; the character picture intercepting unit intercepts time pictures and navigation information pictures in the video image information of longline fishing and the video image information of fish capturing according to a preset position; the character picture processing unit carries out graying processing on the intercepted time picture and navigation information picture by using an integer algorithm and then realizes binarization processing through threshold value conversion; and the picture character extraction unit identifies the picture after binarization processing through an identification tool and extracts time, longitude and latitude, course and speed. The character extraction module further comprises a character recognition training unit, and the character recognition training unit takes each frame in the video image information of longline fishing and the video image information of fish capture as a training sample to train a user-defined word stock.
The character picture intercepting unit opens video data by adopting a VideoCapture () function of opencv-python, and then reads a frame output picture of a monitoring video, wherein the picture resolution of the video output used in the embodiment is 704 multiplied by 576, the horizontal and vertical resolutions are both 96dpi, and the bit depth is 24. In the whole frame, the time picture is intercepted by the frame [19:51,448:676], the navigation information picture (ship name, longitude and latitude, course, speed) is intercepted by the frame [451:572,565:696], and then the picture is saved. Outputting multiple pictures helps to improve recognition accuracy. The sample processing is used for intercepting the picture, so that invalid text boxes are rarely generated in the process of selecting (makebox) the output text boxes, and the manual labeling speed is improved.
In the embodiment, the video frame is read through opencv-python, the part containing characters is intercepted in the set range and stored as the RGB picture (taking the characters in the sailing state of the fishing boat as an example), and in order to improve the accuracy of character recognition, the RGB color picture is converted into a gray scale image (shown in the left side of figure 3) and then is converted into a binary image (shown in the right side of figure 3). In order to avoid low-speed floating point operation, the text image processing unit in this embodiment uses an integer algorithm to realize conversion from an RGB color image to a grayscale image, and enlarges the image by 1000 times to realize integer operation and then divides the image by 1000, where the formula is gray ═ R (R × 299+ G × 587+ B × 114+10) ÷ 1000. By threshold switching
The picture is converted into a binary drawing. In order to distinguish text from background, the font is set to be white, namely, the value is 255, and black sides are added, namely, the value is 0 when text information is embedded in the video.
In the embodiment, the software jTessBoxEditor-2.2.1 for identifying the character training unit is used for collecting a training sample, and the software is developed by Java, so that a Java Running Environment (JRE) needs to be installed. Running a command in DOS opens the interface. A sample picture (see fig. 2), a picture output by merging using Merge TIFF in a jtessBoxEditor software tools menu, and a picture naming format [ lang ] [ fontname ]. exp [ num ]. tif, wherein lang is a language name, fontname is a font name, num is a serial number, and if a picture training custom word library is a document (document time position name) font name, the merged picture is named as document. A character Box, font, exp0, Box file is generated by using the command (1), and the combined picture is opened by using Open in the Box Editor for character correction (see fig. 4). And modifying char according to the content selected by the frame in the picture, and inputting a blank space if the characters are incomplete.
In the folder, a font character text file is newly created, named font _ properties (without extension), opened by a notebook, written in a content format of < fontname > < italic > < bold > < fixed > < serif > < freektur >, and respectively representing the font name, italic, bold, fixed, serif and gothic, and the value is represented by 1 and 0 for presence or absence. For example, "font 00000" takes a value of 0, which indicates that the font is not bold, italic, bold, etc., and the text is used in the command (4) and the command (5).
Generating a character characteristic TR file by using a command (2), extracting characters from all files by using a command (3), generating a unicastet file, gathering character characteristics by using a command (4), generating a unicastet file, generating an mttemp file by using a command (5), and combining all TR files by using a command (6) to generate a norm proto. Five files that will be created during training: shareable, norproto, inttemp, pffmtable, unicarset rename, rename five files with lang as prefix (e.g., vessel.), merge the five files with command (7) to generate vessel.
The above command is in the following specific form:
1 tesserate vector, font, exp0, tif vector, font, exp0, batch, nochop makebox generates character box vector, font, exp0, box
2 tesserate vector, font, exp0.GIF vector, font, exp0 not desk box, train generates TR file vector, font, exp0.TR
3 unicarset _ extra vector, font, exp0, box generates character feature file unicarset
4 shapemetering-F font properties-U unicharaset document
5mftraining-F font properties-U unicaste-O unicaste
6cnt creating document. font. exp0.tr merging all tr files normproto
7combine _ testavessel. merge five files vessel
The picture text extraction unit in this embodiment is programmed by using Python 3.6, the text recognition tool is tesseractv5.0, and the obtained string format is as shown in table 1, by calling pytesseract. The extracted information comprises time, longitude and latitude, course and speed. The Time recorded in the video is UTC (Universal Time coded) Time, and the format is yyy-MM-dd HH: MM: ss; longitude format is ddd degrees mm.mm E (W), ddd value range [0,180], mm.mm value range [0,60) two decimal places, E represents east longitude, W represents west longitude; the latitude format is dd degrees mm.mm N (S), dd value range [0,90], mm.mm value range [0,60) two decimal places, N is cup latitude, and S is south latitude; the course is recorded by direction angles, the format is NC, the horizontal included angle (C) from the north-pointing direction line of a certain point to the navigation direction line of the fishing boat along the clockwise direction is provided, N is an integer value, and the value range is [0,360 ]; the navigation speed format is XKt, X is decimal for reserving 1 digit, the navigation speed of the fishing boat, and the unit knot (Kt).
Table 1 video string format
960 records of 2019-2-2223:00 to 2019-2-2315:00 for 16 hours are extracted at intervals of 1 minute, the data content comprises ship name, longitude, latitude, navigational speed and heading, and no extraction error record is found through comparison with the original picture. FIG. 5 is a trace of the time period, the color is from light to dark to indicate the speed of the boat is from small to large, and the arrow direction indicates the heading of the fishing boat.
The fishing tackle for tuna longline fishing consists of a floating ball, a float rope, a main line, branch lines and a fishhook, wherein the main line is collected and released through a reel and wound on a reel of the reel (shot by a No. 4 camera). When the storage information extraction module of the main line for longline fishing in the embodiment is used for processing, the original RGB color picture is first converted into a gray scale image, and the diameter of the spool axis line is changed along with the change of the winding amount of the main line, so that the radius is the largest when the main line is completely stored (see fig. 6 left), and the radius is the smallest when the main line is paid off (see fig. 6 right). Then, when the brightness difference between the pixel and the surrounding pixels is less than a specific value, a low-pass filter is used to smooth the brightness of the pixel for denoising and blurring, the low-pass filter allows low-frequency signals to pass through, but weakens the passing of signals with frequencies higher than a cut-off frequency, and the embodiment uses Gaussian filtering (GaussianBlur) filtering kernel with the size of (17,17) and comprises a binary group of width and height. Obtaining a final image (see fig. 7) by the difference between the original gray image and the filtered image, wherein a is a point when the spool shaft has a winding main line, and O is a point at the intersection of the spool shaft line and the spool disc; c is a point on the outer ring of the reel disc. Assuming that the diameter of the spool shaft outer disk is M, the diameter of the spool shaft with no thread is N, the length of the spool is K, the number of pixels of the spool OC when the spool shaft with no thread is B, the actual length of the spool is B, the number of pixels of the spool OC during the main thread winding process is N, the total length of the main thread is L, the diameter of the main thread is D, K/D is the number of lines arranged between the two outer disks on the spool, (N × B/B) is the actual length of OC, the length of the main thread of one turn outside the spool is π x (M- (N × B/B) × 2), the length of the main thread of one turn inside the spool is π x N, the number of layers of the main thread wound from inside to outside of the spool is (N × B/B-N) × D, the length of the main thread on the spool is (K/D) × (π x (M- (N × B) × 2) + π N) × ((N B-N) × D), the payout main line length is: l- (K/D) × (π x (M- (N × B/B) x 2) + π N) × ((N × B/B-N) ÷ D/2).
When a crew catches a tuna (shot by a camera No. 1) with a fishing rod with a fish hook and finds the tuna on a branch line on a right side board, one crew catches the tuna in seawater with the fishing rod, drags the tuna onto a deck, and drags the tuna to a front deck for processing, and a video image is shown in FIG. 8. There will be a floating ball during towing of the main line as shown in figure 9. And labeling a label on a tuna capture and floating ball recovery picture, training by using a deep learning model, and extracting time, longitude and latitude positions, navigation speed and course information of the tuna capture and floating ball recovery state by using a character extraction module.
Assuming that the first floating ball recovered in the longline fishing operation is N1, the last floating ball is Nk, and the other middle floating balls are Ni, calculating the length l1 of the releasing line when the floating ball is N1, the length lk of the releasing line of the floating ball Nk and the length li of the releasing line of the middle floating ball Ni according to the longline fishing main line releasing and releasing information extraction module; when the main line pay-off line is at the position of (lon, lat) and the direction angle is alpha, the longitude of the Ni floating ball is loni ═ lon + li × sin α ÷ [ R × cos (lat) × 2 pi ÷ 360], and the latitude is lati ═ lat + li × cos α ÷ [ R × 2 pi ÷ 360], where R is the radius of the earth. The first tuna caught is Fj, the serial numbers of the two floating balls where each tuna is located are recorded, and the position where the tuna hooks is the middle position of the two floating balls. Through the operation process of once putting and taking the net lines, the number of the caught tunas, the length of the line laying and the ratio of the number of the tunas to the length of the line laying can be counted and used as the catching efficiency.
The forward deck crew grasps the branch line, coils it and places it into the barrel for branch line recovery (see fig. 10), thus targeting the combination of man and barrel for branch line recovery. During the unloading and processing of tuna, the front deck contains the tuna (see fig. 11). The method comprises the steps of dividing pictures into categories of floating ball recovery, tuna capture, branch line recovery, tuna processing, tuna unloading and the like, carrying out manual marking on the images, training multiple classified targets, determining a frame where the target is located by adopting target identification, target detection and target positioning technologies, outputting target information, and outputting time, longitude and latitude, navigational speed and course information corresponding to the frame. After the target of the tuna is captured, the captured picture and the video of the tuna in the front and back 9 seconds are captured and are named and stored in a ship name-time-camera number mode, so that the tuna can be conveniently filed and searched.
It is not difficult to find that the invention can obtain the process of longline fishing operation, count the number of the tunas caught, the arrangement condition of the floating balls and the like according to the monitoring video obtained by the monitoring camera in the fishing production process of the longline fishing boat, extract the fishing information based on the monitoring video, assist the fishery production and management, bring convenience to fishery enterprises and enable the fishing boat to be managed orderly and efficiently. Information such as time, position, navigation state, unhooking start-stop time, tuna catching time and the like extracted from the video can be inquired by fishery enterprises, fishing conditions can be counted, and aquatic product catching can be traced; the consumer can inquire and know the catching time and position of the purchased tuna and catch screenshots and images; the fishery management organization can monitor whether illegal fishing exists or not.
Claims (5)
1. The utility model provides a longline fishing information extraction system based on surveillance video which characterized in that includes: the system comprises a video acquisition device and a monitoring host; the video acquisition device is used for acquiring longline fishing video image information and fish capture video image information; the monitoring host comprises a character extraction module, a longline fishing main line receiving and releasing information extraction module and a fish catching information extraction module; the character extraction module is used for intercepting character image areas in the longline fishing video image information and the fish capturing video image information, and extracting character information from the character image areas to obtain information of the position and the navigation state of the fishing boat; the long line fishing main line receiving and releasing information extraction module is used for extracting the length of a main line in the long line fishing video image information; the long line fishing main line receiving and releasing information extraction module carries out the following processing on the long line fishing video image information: firstly, graying the longline video image information, then smoothing the brightness of pixels by using a low-pass filter, then performing Gaussian filtering, and performing difference processing on the grayed picture and the Gaussian filtered picture to obtain a final picture, wherein the point of the intersection of a reel axis line and a reel disc in the final picture is marked as O, the point on the outer ring of the reel disc is marked as C, the pixel number of the reel disc OC when the reel axis is not provided with the line is marked as B, the actual length is B, the length of the main line to be released is obtained by calculating L- (K/D) x (π x (M- (N × B/B) x 2) + π N) x ((N × B/B-N) D/2), wherein L is the total length of the main line, K is the length of the reel, D is the diameter of the main line, M is the diameter of an external disk of the reel shaft, N is the diameter of the reel shaft without the main line, and N is the number of pixels of OC in the process of winding and unwinding the main line by the reel; the fish catching information extraction module is used for extracting the quantity information of the caught fishes in the longline fishing video image information.
2. The system for extracting longline fishing information based on a surveillance video according to claim 1, wherein the video acquiring means includes a first camera, a second camera, a third camera and a fourth camera; the first camera is positioned on the starboard of the fishing boat and used for acquiring image information of long line fishing branch line recovery, fish capture and fish unloading of the fishing boat; the second camera is positioned on the port of the fishing boat and used for acquiring the image information of the fish in and out of the bin; the third camera is positioned at the top of the front deck and is used for acquiring image information of processing of the fish on the deck; and the fourth camera is positioned on the rear deck and used for acquiring the winding and unwinding conditions of the long-line fishing line.
3. The system for extracting information on longline fishing based on a surveillance video as claimed in claim 1, wherein the text extraction module comprises a text picture intercepting unit, a text picture processing unit and a picture text extraction unit; the character picture intercepting unit intercepts time pictures and navigation information pictures in the video image information of longline fishing and the video image information of fish capturing according to a preset position; the character picture processing unit carries out graying processing on the intercepted time picture and navigation information picture by using an integer algorithm and then realizes binarization processing through threshold value conversion; and the picture character extraction unit identifies the picture after binarization processing through an identification tool and extracts time, longitude and latitude, course and speed.
4. The system for extracting longline fishing information based on surveillance video according to claim 1, wherein the character extraction module further comprises a recognition character training unit for training a custom character library using each frame of the image information of the longline fishing video and the image information of the fish capturing video as training samples.
5. The system of claim 1, wherein the fish catching information extraction module labels the pictures of fish catching and float ball retrieving, trains the pictures through a deep learning model, extracts the position and navigation state information of the fishing boat when the fish catching is performed and the length of the main line when the float ball retrieving is performed according to the character extraction module, and counts the number of caught fish, the length of the released line and the catching efficiency through one-time releasing and retrieving of the net line, wherein the first float ball of the long rope fishing operation retrieving is N1, the last float ball is Nk, the middle float ball is Ni, the position when the main line releasing is finished is (lon, lat), the length of the middle float ball Ni releasing line is li, the direction angle is α, the longitude of the float ball Ni is loni + lin + li x α ÷ [ R x costas lat x 2 pi ],360, the latitude is lati ═ lat + li × cos α ÷ [ R × 2 pi ÷ 360], and R is the earth radius.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010050955.3A CN111263111B (en) | 2020-01-17 | 2020-01-17 | Long line fishing information extraction system based on surveillance video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010050955.3A CN111263111B (en) | 2020-01-17 | 2020-01-17 | Long line fishing information extraction system based on surveillance video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111263111A CN111263111A (en) | 2020-06-09 |
| CN111263111B true CN111263111B (en) | 2021-03-12 |
Family
ID=70952269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010050955.3A Active CN111263111B (en) | 2020-01-17 | 2020-01-17 | Long line fishing information extraction system based on surveillance video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111263111B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112214717B (en) * | 2020-10-23 | 2021-07-02 | 中国水产科学研究院东海水产研究所 | Method for estimating catching depth by utilizing sonar data |
| CN113205283B (en) * | 2021-06-02 | 2023-09-12 | 中国水产科学研究院南海水产研究所 | Open sea fishery fishing scheduling method and system |
| CN113569971B (en) * | 2021-08-02 | 2022-03-25 | 浙江索思科技有限公司 | Image recognition-based catch target classification detection method and system |
| CN117809313A (en) * | 2024-02-29 | 2024-04-02 | 江苏中威科技软件系统有限公司 | Positioning device for OCR text extraction |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008212060A (en) * | 2007-03-05 | 2008-09-18 | Masabumi Nishigori | Organism-capturing tool and underwater camera for the organism-capturing tool |
| CN201117017Y (en) * | 2007-04-29 | 2008-09-17 | 上海海事大学 | Ship circumjacent water area safety visible monitoring system |
| CN108288230A (en) * | 2018-01-22 | 2018-07-17 | 无锡职业技术学院 | The fishery system of long range fishing operation is carried out using Internet of Things big data |
| JP6715281B2 (en) * | 2018-03-23 | 2020-07-01 | Ｈａｐｓモバイル株式会社 | Underwater exploration system and information processing device |
| CN109644956A (en) * | 2019-01-10 | 2019-04-19 | 上海和创船舶工程有限公司 | A kind of fish finding unmanned boat |
-
2020
- 2020-01-17 CN CN202010050955.3A patent/CN111263111B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN111263111A (en) | 2020-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111263111B (en) | Long line fishing information extraction system based on surveillance video | |
| Yang et al. | Computer vision models in intelligent aquaculture with emphasis on fish detection and behavior analysis: a review | |
| Pedersen et al. | Detection of marine animals in a new underwater dataset with varying visibility | |
| Harvey et al. | The use of BRUVs as a tool for assessing marine fisheries and ecosystems: a review of the hurdles and potential | |
| DeCelles et al. | Development of a video trawl survey system for New England groundfish | |
| CN113591592B (en) | Overwater target identification method and device, terminal equipment and storage medium | |
| CN112036206A (en) | Monitoring method, device and system based on identification code type pig ear tag | |
| CN112784180A (en) | Method for extracting catching strength spatial information of tuna seine fishing boat | |
| CN116778310A (en) | Acoustic-optical image fusion monitoring method and system for aquaculture | |
| CN115578423A (en) | Fish key point detection, individual tracking and biomass estimation method and system based on deep learning | |
| CN113705504A (en) | Marine fishery safety production management system based on video processing technology | |
| Underwood et al. | Development and evaluation of a new high definition self-contained underwater camera system to observe fish and fishing gear in situ | |
| CN101620420B (en) | control system for steel industry | |
| Neidig et al. | Employing an innovative underwater camera to improve electronic monitoring in the commercial Gulf of Mexico reef fish fishery | |
| CN114332682B (en) | Marine panorama defogging target identification method | |
| CN109344691A (en) | A kind of visual security campus administration monitoring method based on deep learning | |
| CN209845144U (en) | Seabed polymetallic nodule image real-time processing system | |
| McElderry et al. | Pilot study to test the effectiveness of electronic monitoring in Canterbury fisheries | |
| Betti et al. | Real-time target detection in maritime scenarios based on YOLOv3 model | |
| Tuck et al. | Estimating the abundance of scampi in SCI 1 (Bay of Plenty) and SCI 2 (Wairarapa/Hawke Bay) in 2015 | |
| Kelasidi et al. | Cagereporter-development of technology for autonomous, bio-interactive and high-quality data acquisition from aquaculture net cages | |
| CN112532938A (en) | Video monitoring system based on big data technology | |
| CN113869149A (en) | Big data application node service system | |
| CN118169140A (en) | System and method for detecting scratches of towing line array sheath | |
| Walther et al. | Automated video analysis for oceanographic research |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |