WO2022062242A1 - 一种基于深度学习的水下成像鱼网破损识别方法及系统 - Google Patents

一种基于深度学习的水下成像鱼网破损识别方法及系统 Download PDF

Info

Publication number
WO2022062242A1
WO2022062242A1 PCT/CN2020/140287 CN2020140287W WO2022062242A1 WO 2022062242 A1 WO2022062242 A1 WO 2022062242A1 CN 2020140287 W CN2020140287 W CN 2020140287W WO 2022062242 A1 WO2022062242 A1 WO 2022062242A1
Authority
WO
WIPO (PCT)
Prior art keywords
fishnet
underwater
damage
damage identification
net
Prior art date
Application number
PCT/CN2020/140287
Other languages
English (en)
French (fr)
Inventor
侯明鑫
俞国燕
王林
梁贻察
何泰华
李军
Original Assignee
广东海洋大学
南方海洋科学与工程广东省实验室(湛江)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广东海洋大学, 南方海洋科学与工程广东省实验室(湛江) filed Critical 广东海洋大学
Publication of WO2022062242A1 publication Critical patent/WO2022062242A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/05—Underwater scenes
    • A—HUMAN NECESSITIES
    • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
    • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
    • A01K75/00—Accessories for fishing nets; Details of fishing nets, e.g. structure
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G06F18/24137—Distances to cluster centroïds
    • G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the invention relates to the technical field of image recognition, and more particularly, to a deep learning-based method and system for recognizing damage to underwater imaging fishnets.
  • Object detection is the study of locating and classifying target objects. With the continuous development of digital image processing, object detection has become one of the key research directions of current researchers. Image recognition technology has been widely used in the fields of medicine, military, industry and agriculture. At present, there is no relevant research applied to the recognition technology of fishnet damage.
  • the convolutional neural network CNN and the deep convolutional neural network DCNN are mainly used in the damage detection based on the deep learning method, and the use of the CNN method can make up for the shortcomings of the traditional methods in terms of accuracy and detection efficiency (Xu Zhigang, Che Yanli). , Li Jinlong, et al. Research progress in automatic processing technology of pavement damage images [J].
  • the present invention provides a deep learning-based underwater imaging fishnet damage identification method, and a deep learning-based method Cage underwater imaging fishnet breakage identification system.
  • the technical scheme of the present invention is as follows:
  • a deep learning-based underwater imaging fishnet damage identification method comprising the following steps:
  • S1 Collect underwater net breaking images to form an image database, and divide the underwater net breaking images in the image database into a training set and a test set; label the underwater net breaking images respectively with their net breaking positions, and label the marked net breaking positions. Data is generated into table files for storage;
  • S3 Adjust the parameters of the fishnet damage identification model according to the configuration file and the classification label directory, and then input the training set into the fishnet damage identification model for training, and obtain the fishnet damage identification model after training;
  • step S4 Input the test set into the trained fishnet damage identification model, output the fishnet damage identification result corresponding to the test set, and compare the fishnet damage identification result with the net damage location data stored in the table file to verify the accuracy of the model: if accurate When the accuracy is greater than or equal to the preset accuracy threshold, step S5 is performed, otherwise, step S1 is performed;
  • S5 Use an underwater camera to collect an underwater fishnet image, input the underwater fishnet image into the trained fishnet damage recognition model, and output the fishnet damage recognition result.
  • the fishnet damage identification model adopts a deep learning model among SSD_MobileNet, YOLO, SSD_Inception, and R-FCN_ResNet.
  • the specific steps of marking the underwater net breaking images respectively with their net breaking positions include: constructing an image coordinate system in the underwater net breaking images; marking the circumscribed rectangles of the net breaking positions in the underwater net breaking images
  • the frame is used as the real frame, and the center coordinates of the real frame and the frame size, the damage type in the underwater mesh image are recorded as the real mesh information, and the real mesh data is generated into a table file for storage.
  • step S3 its specific steps include:
  • the fishnet damage recognition model classifies the damage types in the underwater mesh image according to the classification label catalog, and outputs the fishnet damage types in the underwater mesh image;
  • step S3 when the fishnet damage identification model identifies the position of the net break in the underwater net break image, a total of N default boxes are generated; when the fish net break identification model classifies the damage types in the underwater net break image, calculate The predicted value of the classification confidence c.
  • the loss function L is the sum of the fishnet net breaking location loss function Lloc and the fishnet net breaking classification loss function L conf , and its expression formula is as follows:
  • l represents the predicted frame value
  • g represents the real frame value
  • x represents the network prediction value of the fishnet damage recognition model
  • the expression formula of the fishnet damage classification loss function L conf is as follows:
  • (cx, cy) represents the center coordinate of the prediction frame
  • w represents the width of the prediction frame
  • h represents the height of the prediction frame
  • smooth L1 ( ) represents the smoothing coefficient
  • the preset accuracy threshold value ranges from 90% to 95%.
  • the present invention also provides an underwater imaging fishnet damage identification system based on deep learning, which is applied to the deep learning-based underwater imaging fishnet damage identification method proposed by any of the above technical solutions, including an imaging module and a microcomputer, wherein: imaging The module is used to collect underwater fishnet images, and the output end of the imaging module is connected to the input end of the microcomputer; the microcomputer is embedded with a trained fishnet damage recognition model, which is used to identify the fishnet damage to the collected underwater fishnet images, and output Obtain the fishnet damage recognition result; the fishnet damage recognition result includes the image marked with the fishnet damage recognition prediction frame, and the fishnet damage category.
  • the microcomputer is a Raspberry Pi microcomputer.
  • the system further includes a communication module, the output end of the microcomputer is connected to the input end of the communication module; the communication module is used to send the collected underwater fishnet image and the fishnet damage identification result output by the microcomputer to the communication terminal of the staff.
  • the beneficial effects of the technical solution of the present invention are: the present invention performs fishnet damage identification on the collected underwater fishnet images through the fishnet damage identification model that has completed the training, and obtains the fishnet damage identification result, thereby avoiding the occurrence of cultured fish from being damaged.
  • the damaged parts escape from the breeding area and reduce the economic losses of fishermen; the SSD_MobileNet lightweight deep learning model is used to build a fishnet damage identification model, so that it can be used on the engineering carrier of underwater operations.
  • FIG. 1 is a flowchart of the deep learning-based underwater imaging fishnet damage identification method according to Embodiment 1.
  • FIG. 1 is a flowchart of the deep learning-based underwater imaging fishnet damage identification method according to Embodiment 1.
  • FIG. 2 is a schematic structural diagram of a deep learning-based underwater imaging fishnet damage identification system according to Embodiment 2.
  • FIG. 2 is a schematic structural diagram of a deep learning-based underwater imaging fishnet damage identification system according to Embodiment 2.
  • This embodiment proposes a deep learning-based underwater imaging fishnet damage identification method, as shown in FIG. 1 , which is a flowchart of the deep learning-based underwater imaging fishnet damage identification method of this embodiment.
  • S1 Collect underwater net breaking images to form an image database, and divide the underwater net breaking images in the image database into a training set and a test set; label the underwater net breaking images respectively with their net breaking positions, and label the marked net breaking positions.
  • the data is stored in tabular files.
  • the specific steps of marking the position of the underwater net breaking image respectively include: constructing an image coordinate system in the underwater net breaking image; marking the circumscribed rectangular frame of the net breaking position in the underwater net breaking image as the real frame, record the center coordinates of the real frame, the size of the frame, and the damage type in the underwater net breaking image as the real net breaking information, and generate a table file for the real net breaking data for storage.
  • the fishnet damage identification model adopts a deep learning model among SSD_MobileNet, YOLO, SSD_Inception, and R-FCN_ResNet.
  • the SSD_MobileNet deep learning model is adopted.
  • S3 Adjust the parameters of the fishnet damage identification model according to the configuration file and the classification label directory, and then input the training set into the fishnet damage identification model for training, and obtain the fishnet damage identification model after training.
  • the specific steps are as follows:
  • the fishnet damage recognition model classifies the damage types in the underwater mesh image according to the classification label catalog, and outputs the fishnet damage types in the underwater mesh image;
  • the loss function L is the sum of the fishnet and net breaking location loss function Lloc and the fishnet net breaking classification loss function L conf , and its expression formula is as follows:
  • L conf the expression formula of the fishnet damage classification loss function L conf is as follows:
  • (cx, cy) represents the center coordinate of the prediction frame
  • w represents the width of the prediction frame
  • h represents the height of the prediction frame
  • smooth L1 ( ) represents the smoothing coefficient
  • step S4 Input the test set into the trained fishnet damage identification model, output the fishnet damage identification result corresponding to the test set, and compare the fishnet damage identification result with the net damage location data stored in the table file to verify the accuracy of the model: if accurate When the accuracy is greater than or equal to the preset accuracy threshold, step S5 is performed, otherwise, step S1 is performed.
  • the preset accuracy threshold value ranges from 90% to 95%, and in this embodiment, the set accuracy threshold value is 95%.
  • S5 Use an underwater camera to collect an underwater fishnet image, input the underwater fishnet image into the trained fishnet damage recognition model, and output the fishnet damage recognition result.
  • the SSD_MobileNet lightweight deep learning model is used to build a fishnet damage identification model, so that it can be mounted on an engineering carrier for underwater operations.
  • it is possible to identify the broken part of the net for underwater aquaculture in cages in a timely manner, so as to avoid the escape of cultured fish from the damaged part of the aquaculture area and reduce the economic losses of fishermen.
  • This embodiment proposes a deep learning-based underwater imaging fishnet damage identification system, which is applied to the deep learning-based underwater imaging fishnet damage identification method proposed in Embodiment 1.
  • FIG. 2 it is a schematic structural diagram of the deep learning-based underwater imaging fishnet damage identification system of this embodiment.
  • the deep learning-based underwater imaging fishnet damage recognition system proposed in this embodiment includes an imaging module 1 and a microcomputer 2, wherein: the imaging module 1 is used to collect underwater fishnet images, and the output end of the imaging module 1 is connected to the microcomputer 2
  • the microcomputer 2 is embedded with a trained fishnet damage identification model, which is used to identify the fishnet damage to the collected underwater fishnet images, and output the fishnet damage identification result; the fishnet damage identification result includes the fishnet damage identification marked with The image of the predicted box, and the fishnet breakage category.
  • the imaging module 1 adopts an underwater camera
  • the microcomputer 2 adopts a Raspberry Pi microcomputer 2 .
  • the system further includes a communication module 3, and the output end of the microcomputer 2 is connected to the input end of the communication module 3; to the communication terminal of the staff.
  • the communication module 3 in this embodiment uses underwater wireless communication technologies such as radio frequency technology, underwater acoustic communication, and underwater quantum communication to perform the data communication.
  • the imaging module 1 is placed underwater to collect underwater fishnet images, and the imaging module 1 transmits the collected underwater fishnet images to the microcomputer 2 for identification and analysis, wherein the microcomputer 2 is equipped with A trained fishnet damage identification model is preset, and the collected underwater fishnet image is input into the trained fishnet damage identification model, and the model outputs the fishnet damage identification result.
  • the fishnet damage identification result includes the fishnet damage identification marked with the fishnet damage identification. The image of the predicted box, and the fishnet breakage category.
  • the microcomputer 2 sends the fishnet damage identification result obtained through the communication module 3 to the communication terminal of the staff member, and the staff member can check the fishnet damage identification result through the communication terminal, and further judge the broken net part of the underwater culture in the cage.
  • the deep learning-based underwater imaging fishnet damage identification system proposed in this embodiment can be applied to application carriers such as underwater remote-controlled unmanned submersibles, cableless underwater robots, deep-sea landers, and marine robots.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Multimedia (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Library & Information Science (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Environmental Sciences (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Animal Husbandry (AREA)
  • Marine Sciences & Fisheries (AREA)
  • Image Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • Farming Of Fish And Shellfish (AREA)

Abstract

本发明提出一种基于深度学习的水下成像鱼网破损识别方法，包括以下步骤：采集图像数据库并划分为训练集和测试集；对水下网破图像分别进行标注其网破位置并生成表格文件进行存储；构建鱼网破损识别模型；将训练集输入鱼网破损识别模型中进行训练，得到完成训练的鱼网破损识别模型；将测试集输入完成训练的鱼网破损识别模型中，输出测试集对应的鱼网破损识别结果，并验证模型的精准度，得到完成训练的鱼网破损识别模型；采用水下摄像头采集水下鱼网图像，将水下鱼网图像输入完成训练的鱼网破损识别模型中，输出得到鱼网破损识别结果。本发明还提出了一种基于深度学习的水下成像鱼网破损识别系统，能够搭载在水下作业的工程载体上使用。

Description

一种基于深度学习的水下成像鱼网破损识别方法及系统 技术领域
本发明涉及图像识别技术领域，更具体地，涉及一种基于深度学习的水下成像鱼网破损识别方法及系统。
背景技术
目标检测是对目标物体进行定位和分类的研究，随着数字图像处理的不断发展，目标检测成为当下研究者的重点研究方向之一。图像识别技术已被广泛应用于医学、军事、工业和农业领域，目前，尚未有应用于鱼网破损识别技术的相关研究。
相近的，研究者徐志刚等提出了路面破损图像自动处理技术，在对路面破损进行目标检测时，主要采用阈值分割法、边缘检测法、基于多尺度的裂缝检测方法、基于纹理的分割方法、基于多特征融合的方法、基于图论的分割方法、基于深度学习的方法等。其中，在基于深度学习的方法进行破损检测时，主要采用卷积神经网络CNN、深度卷积神经网络DCNN，且使用CNN方法能够弥补传统方法在准确率和检测效率方面的不足(徐志刚,车艳丽,李金龙,et al.路面破损图像自动处理技术研究进展[J].交通运输工程学报,2019,19(01):176-194.)。然而，在应用于水下成像鱼网破损识别领域时，直接采用卷积神经网络CNN、深度卷积神经网络DCNN会存在算法较复杂，难以搭载在水下作业的工程载体上使用。
发明内容
本发明为克服上述现有技术所述的算法复杂、难以搭载在水下作业的工程载体上使用的缺陷，提供一种基于深度学习的水下成像鱼网破损识别方法，以及一种基于深度学习的网箱水下成像鱼网破损识别系统。
为解决上述技术问题，本发明的技术方案如下：
一种基于深度学习的水下成像鱼网破损识别方法，包括以下步骤：
S1：采集水下网破图像组成图像数据库，将图像数据库中的水下网破图像划分为训练集和测试集；对水下网破图像分别进行标注其网破位置，将标注的网破位置数据生成表格文件进行存储；
S2：构建鱼网破损识别模型，选取鱼网破损识别模型训练需要的配置文件，并预设鱼网破损识别模型的分类标签目录；
S3：根据配置文件及分类标签目录调整鱼网破损识别模型的参数，然后将训练集输入鱼网破损识别模型中进行训练，得到完成训练的鱼网破损识别模型；
S4：将测试集输入完成训练的鱼网破损识别模型中，输出测试集对应的鱼网破损识别结果，将鱼网破损识别结果与表格文件中存储的网破位置数据进行对比验证模型的精准度：若精准度大于或等于预设的精准度阈值时，则执行S5步骤，否则跳转执行S1步骤；
S5：采用水下摄像头采集水下鱼网图像，将水下鱼网图像输入完成训练的鱼网破损识别模型中，输出得到鱼网破损识别结果。
优选地，鱼网破损识别模型采用SSD_MobileNet、YOLO、SSD_Inception、R-FCN_ResNet中的一种深度学习模型。
优选地，S1步骤中，对水下网破图像分别进行标注其网破位置的具体步骤包括：在水下网破图像中构建图像坐标系；在水下网破图像中网破位置标注外接矩形框作为真实框，记录真实框的中心坐标及框体尺寸、水下网破图像中破损类型作为真实网破信息，并将真实网破数据生成表格文件进行存储。
优选地，S3步骤中，其具体步骤包括：
S3.1：根据配置文件及分类标签目录调整鱼网破损识别模型的参数；
S3.2：将训练集中的水下网破图像依次输入鱼网破损识别模型中，鱼网破损识别模型识别水下网破图像中的网破位置并标注预测框，输出标注有预测框的水下网破图像，以及预测框的中心坐标及框体尺寸；
S3.3：鱼网破损识别模型对水下网破图像中的破损类型根据分类标签目录进行分类，输出水下网破图像中鱼网破损类型；
S3.4：将鱼网破损识别模型输出的预测框的中心坐标及框体尺寸、鱼网破损类型，与对应水下网破图像中的真实网破数据进行对比，计算鱼网破损识别模型的损失函数，并根据损失函数的结果对鱼网破损识别模型的参数进行优化，得到完成训练的鱼网破损识别模型。
优选地，S3步骤中，鱼网破损识别模型识别水下网破图像中的网破位置时，共生成N个默认框；鱼网破损识别模型对水下网破图像中的破损类型进行分类时，计算分类置信度的预测值c。
优选地，S3步骤中，损失函数L为鱼网网破定位损失函数L loc和鱼网网破分类损失函数L conf之和，其表达公式如下：
Figure PCTCN2020140287-appb-000001
其中，l表示预测框值，g表示真实框值，x表示鱼网破损识别模型网络预测值；
Figure PCTCN2020140287-appb-000002
表示鱼网破损识别模型预设值；鱼网网破分类损失函数L conf的表达公式如下：
Figure PCTCN2020140287-appb-000003
Figure PCTCN2020140287-appb-000004
其中，
Figure PCTCN2020140287-appb-000005
表示当鱼网网破预测框i与真实框j关于类别p匹配时的概率预测值；鱼网网破定位损失函数L loc的表达公式如下：
Figure PCTCN2020140287-appb-000006
其中，(cx,cy)表示预测框的中心坐标，w表示预测框的宽度，h表示预测框的高度；
Figure PCTCN2020140287-appb-000007
表示预测框，
Figure PCTCN2020140287-appb-000008
表示真实框；smooth L1(·)表示平滑系数。
优选地，S4步骤中，预设的精准度阈值的取值范围为90％～95％。
本发明还提出了一种基于深度学习的水下成像鱼网破损识别系统，应用于上述任一技术方案提出的基于深度学习的水下成像鱼网破损识别方法，包括成像模块和微型电脑，其中：成像模块用于采集水下鱼网图像，成像模块的输出端与微型电脑的输入端连接；微型电脑嵌入设置有完成训练的鱼网破损识别模型，用于对采集的水下鱼网图像进行鱼网破损识别，输出得到鱼网破损识别结果；鱼网破损识别结果包括标注有鱼网破损识别预测框的图像，以及鱼网破损类别。
优选地，微型电脑采用树莓派微型电脑。
优选地，系统还包括通信模块，微型电脑的输出端与通信模块的输入端连接；通信模块用于将采集的水下鱼网图像以及微型电脑输出的鱼网破损识别结果发送至工作人员的通信终端。
与现有技术相比，本发明技术方案的有益效果是：本发明通过完成训练的鱼网破损识别模型对采集的水下鱼网图像进行鱼网破损识别，得到鱼网破损识别结 果，从而避免出现养殖鱼从破损部位逃出养殖区域，减少渔民的经济损失；采用SSD_MobileNet轻量级深度学习模型构建鱼网破损识别模型，使其能够搭载在水下作业的工程载体上使用。
附图说明
图1为实施例1的基于深度学习的水下成像鱼网破损识别方法的流程图。
图2为实施例2的基于深度学习的水下成像鱼网破损识别系统的结构示意图。
具体实施方式
附图仅用于示例性说明，不能理解为对本专利的限制；
为了更好说明本实施例，附图某些部件会有省略、放大或缩小，并不代表实际产品的尺寸；
对于本领域技术人员来说，附图中某些公知结构及其说明可能省略是可以理解的。
下面结合附图和实施例对本发明的技术方案做进一步的说明。
实施例1
本实施例提出一种基于深度学习的水下成像鱼网破损识别方法，如图1所示，为本实施例的基于深度学习的水下成像鱼网破损识别方法的流程图。
本实施例提出的基于深度学习的水下成像鱼网破损识别方法中，包括以下步骤：
S1：采集水下网破图像组成图像数据库，将图像数据库中的水下网破图像划分为训练集和测试集；对水下网破图像分别进行标注其网破位置，将标注的网破位置数据生成表格文件进行存储。
本步骤中，对水下网破图像分别进行标注其网破位置的具体步骤包括：在水下网破图像中构建图像坐标系；在水下网破图像中网破位置标注外接矩形框作为真实框，记录真实框的中心坐标及框体尺寸、水下网破图像中破损类型作为真实网破信息，并将真实网破数据生成表格文件进行存储。
S2：构建鱼网破损识别模型，选取鱼网破损识别模型训练需要的配置文件，并预设鱼网破损识别模型的分类标签目录。其中，鱼网破损识别模型采用SSD_MobileNet、YOLO、SSD_Inception、R-FCN_ResNet中的一种深度学习模型，本实施例中，采用SSD_MobileNet深度学习模型。
S3：根据配置文件及分类标签目录调整鱼网破损识别模型的参数，然后将训练集输入鱼网破损识别模型中进行训练，得到完成训练的鱼网破损识别模型。其具体步骤如下：
S3.1：根据配置文件及分类标签目录调整鱼网破损识别模型的参数；
S3.2：将训练集中的水下网破图像依次输入鱼网破损识别模型中，鱼网破损识别模型识别水下网破图像中的网破位置并标注预测框，输出标注有预测框的水下网破图像，以及预测框的中心坐标及框体尺寸；
S3.3：鱼网破损识别模型对水下网破图像中的破损类型根据分类标签目录进行分类，输出水下网破图像中鱼网破损类型；
S3.4：将鱼网破损识别模型输出的预测框的中心坐标及框体尺寸、鱼网破损类型，与对应水下网破图像中的真实网破数据进行对比，计算鱼网破损识别模型的损失函数，并根据损失函数的结果对鱼网破损识别模型的参数进行优化，得到完成训练的鱼网破损识别模型。
本步骤中，在鱼网破损识别模型识别水下网破图像中的网破位置时，共生成N个默认框；鱼网破损识别模型对水下网破图像中的破损类型进行分类时，计算分类置信度的预测值c。
本步骤中，损失函数L为鱼网网破定位损失函数L loc和鱼网网破分类损失函数L conf之和，其表达公式如下：
Figure PCTCN2020140287-appb-000009
其中，l表示预测框值，g表示真实框值，x表示鱼网破损识别模型网络预测值；鱼网网破分类损失函数L conf的表达公式如下：
Figure PCTCN2020140287-appb-000010
Figure PCTCN2020140287-appb-000011
其中，
Figure PCTCN2020140287-appb-000012
表示当鱼网网破预测框i与真实框j关于类别p匹配时的概率预测值；
Figure PCTCN2020140287-appb-000013
表示鱼网破损识别模型预设值；鱼网网破定位损失函数L loc的表达公式如下：
Figure PCTCN2020140287-appb-000014
其中，(cx,cy)表示预测框的中心坐标，w表示预测框的宽度，h表示预测框的高度；
Figure PCTCN2020140287-appb-000015
表示预测框，
Figure PCTCN2020140287-appb-000016
表示真实框；smooth L1(·)表示平滑系数。
S4：将测试集输入完成训练的鱼网破损识别模型中，输出测试集对应的鱼网破损识别结果，将鱼网破损识别结果与表格文件中存储的网破位置数据进行对比验证模型的精准度：若精准度大于或等于预设的精准度阈值时，则执行S5步骤，否则跳转执行S1步骤。
其中，预设的精准度阈值的取值范围为90％～95％，在本实施例中，设置精准度阈值为95％。
S5：采用水下摄像头采集水下鱼网图像，将水下鱼网图像输入完成训练的鱼网破损识别模型中，输出得到鱼网破损识别结果。
本实施例中，为了适用于水下成像及鱼网破损识别，采用SSD_MobileNet轻量级深度学习模型构建鱼网破损识别模型，使其能够搭载在水下作业的工程载体上使用。在实际应用中，可针对网箱水下渔业养殖网破问题，及时识别出网箱水下养殖的网破部分，避免出现养殖鱼从破损部位逃出养殖区域，减少渔民的经济损失。
实施例2
本实施例提出一种基于深度学习的水下成像鱼网破损识别系统，应用于实施例1提出的基于深度学习的水下成像鱼网破损识别方法。如图2所示，为本实施例的基于深度学习的水下成像鱼网破损识别系统的结构示意图。
本实施例提出的基于深度学习的水下成像鱼网破损识别系统中，包括成像模块1和微型电脑2，其中：成像模块1用于采集水下鱼网图像，成像模块1的输出端与微型电脑2的输入端连接；微型电脑2嵌入设置有完成训练的鱼网破损识别模型，用于对采集的水下鱼网图像进行鱼网破损识别，输出得到鱼网破损识别结果；鱼网破损识别结果包括标注有鱼网破损识别预测框的图像，以及鱼网破损类别。
本实施例中，成像模块1采用水下摄像头，微型电脑2采用树莓派微型电脑2。
本实施例中，系统还包括通信模块3，微型电脑2的输出端与通信模块3的 输入端连接；通信模块3用于将采集的水下鱼网图像以及微型电脑2输出的鱼网破损识别结果发送至工作人员的通信终端。本实施例中的通信模块3采用射频技术、水声通信、水下量子通信等水下无线通信技术进行该数据通信。
在具体实施过程中，成像模块1放置在水下进行水下鱼网图像的采集，成像模块1将其所采集的水下鱼网图像传输至微型电脑2中进行识别分析，其中，微型电脑2中搭载预设有完成训练的鱼网破损识别模型，将所采集的水下鱼网图像输入该完成训练的鱼网破损识别模型，模型输出得到鱼网破损识别结果，具体的，鱼网破损识别结果包括标注有鱼网破损识别预测框的图像，以及鱼网破损类别。微型电脑2将其输出得到的鱼网破损识别结果通过通信模块3发送至工作人员的通信终端，工作人员可以通过其通信终端查看鱼网破损识别结果，进一步判断网箱水下养殖的网破部分。
此外，本实施例提出的基于深度学习的水下成像鱼网破损识别系统可应用于水下遥控无人潜水器、无缆水下机器人、深海着陆器、海洋机器人等应用载体。
相同或相似的标号对应相同或相似的部件；
附图中描述位置关系的用语仅用于示例性说明，不能理解为对本专利的限制；
显然，本发明的上述实施例仅仅是为清楚地说明本发明所作的举例，而并非是对本发明的实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等，均应包含在本发明权利要求的保护范围之内。

Claims (10)

  1. 一种基于深度学习的水下成像鱼网破损识别方法，其特征在于，包括以下步骤：
    S1：采集水下网破图像组成图像数据库，将所述图像数据库中的水下网破图像划分为训练集和测试集；对所述水下网破图像分别进行标注其网破位置，将所述标注的网破位置数据生成表格文件进行存储；
    S2：构建鱼网破损识别模型，选取所述鱼网破损识别模型训练需要的配置文件，并预设所述鱼网破损识别模型的分类标签目录；
    S3：根据所述配置文件及分类标签目录调整所述鱼网破损识别模型的参数，然后将所述训练集输入所述鱼网破损识别模型中进行训练，得到完成训练的鱼网破损识别模型；
    S4：将所述测试集输入所述完成训练的鱼网破损识别模型中，输出所述测试集对应的鱼网破损识别结果，将所述鱼网破损识别结果与所述表格文件中存储的网破位置数据进行对比验证模型的精准度：若精准度大于或等于预设的精准度阈值时，则执行S5步骤，否则跳转执行S1步骤；
    S5：采用水下摄像头采集水下鱼网图像，将所述水下鱼网图像输入所述完成训练的鱼网破损识别模型中，输出得到鱼网破损识别结果。
  2. 根据权利要求1所述的水下成像鱼网破损识别方法，其特征在于：所述鱼网破损识别模型采用SSD_MobileNet、YOLO、SSD_Inception、R-FCN_ResNet中的一种深度学习模型。
  3. 根据权利要求1所述的水下成像鱼网破损识别方法，其特征在于：所述S1步骤中，对所述水下网破图像分别进行标注其网破位置的具体步骤包括：在所述水下网破图像中构建图像坐标系；在所述水下网破图像中网破位置标注外接矩形框作为真实框，记录所述真实框的中心坐标及框体尺寸、所述水下网破图像中破损类型作为真实网破信息，并将所述真实网破数据生成表格文件进行存储。
  4. 根据权利要求3所述的水下成像鱼网破损识别方法，其特征在于：所述S3步骤中，其具体步骤包括：
    S3.1：根据所述配置文件及分类标签目录调整所述鱼网破损识别模型的参数；
    S3.2：将所述训练集中的水下网破图像依次输入所述鱼网破损识别模型中，所述鱼网破损识别模型识别所述水下网破图像中的网破位置并标注预测框，输出标注有预测框的水下网破图像，以及所述预测框的中心坐标及框体尺寸；
    S3.3：所述鱼网破损识别模型对所述水下网破图像中的破损类型根据所述分类标签目录进行分类，输出所述水下网破图像中鱼网破损类型；
    S3.4：将所述鱼网破损识别模型输出的预测框的中心坐标及框体尺寸、鱼网破损类型，与对应水下网破图像中的真实网破数据进行对比，计算所述鱼网破损识别模型的损失函数，并根据所述损失函数的结果对所述鱼网破损识别模型的参数进行优化，得到完成训练的鱼网破损识别模型。
  5. 根据权利要求4所述的水下成像鱼网破损识别方法，其特征在于：所述S3步骤中，所述鱼网破损识别模型识别所述水下网破图像中的网破位置时，共生成N个默认框；所述鱼网破损识别模型对所述水下网破图像中的破损类型进行分类时，计算分类置信度的预测值c。
  6. 根据权利要求5所述的水下成像鱼网破损识别方法，其特征在于：所述S3步骤中，所述损失函数L为鱼网网破定位损失函数L loc和鱼网网破分类损失函数L conf之和，其表达公式如下：
    Figure PCTCN2020140287-appb-100001
    其中，l表示预测框值，g表示真实框值，x表示鱼网破损识别模型网络预测值；鱼网网破分类损失函数L conf的表达公式如下：
    Figure PCTCN2020140287-appb-100002
    Figure PCTCN2020140287-appb-100003
    其中，
    Figure PCTCN2020140287-appb-100004
    表示当鱼网网破预测框i与真实框j关于类别p匹配时的概率预测值；
    Figure PCTCN2020140287-appb-100005
    为鱼网破损识别模型预设值；鱼网网破定位损失函数L loc的表达公式如下：
    Figure PCTCN2020140287-appb-100006
    其中，(cx,cy)表示预测框的中心坐标，w表示预测框的宽度，h表示预测框的高度；
    Figure PCTCN2020140287-appb-100007
    表示预测框，
    Figure PCTCN2020140287-appb-100008
    表示真实框；smooth L1(·)表示平滑系数。
  7. 根据权利要求1所述的水下成像鱼网破损识别方法，其特征在于：所述S4步骤中，所述预设的精准度阈值的取值范围为90％～95％。
  8. 一种基于深度学习的水下成像鱼网破损识别系统，应用于权利要求1～7所述的基于深度学习的水下成像鱼网破损识别方法，其特征在于，包括成像模块和微型电脑，其中：
    所述成像模块用于采集水下鱼网图像，所述成像模块的输出端与所述微型电脑的输入端连接；
    所述微型电脑嵌入设置有完成训练的鱼网破损识别模型，用于对采集的水下鱼网图像进行鱼网破损识别，输出得到鱼网破损识别结果；所述鱼网破损识别结果包括标注有鱼网破损识别预测框的图像，以及鱼网破损类别。
  9. 根据权利要求8所述的水下成像鱼网破损识别系统，其特征在于：所述微型电脑采用树莓派微型电脑。
  10. 根据权利要求8所述的水下成像鱼网破损识别系统，其特征在于：所述系统还包括通信模块，所述微型电脑的输出端与所述通信模块的输入端连接；所述通信模块用于将采集的水下鱼网图像以及所述微型电脑输出的鱼网破损识别结果发送至工作人员的通信终端。
PCT/CN2020/140287 2020-09-27 2020-12-28 一种基于深度学习的水下成像鱼网破损识别方法及系统 WO2022062242A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011036004.7 2020-09-27
CN202011036004.7A CN112163517A (zh) 2020-09-27 2020-09-27 一种基于深度学习的水下成像鱼网破损识别方法及系统

Publications (1)

Publication Number Publication Date
WO2022062242A1 true WO2022062242A1 (zh) 2022-03-31

Family

ID=73861310

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/140287 WO2022062242A1 (zh) 2020-09-27 2020-12-28 一种基于深度学习的水下成像鱼网破损识别方法及系统

Country Status (3)

Country Link
CN (1) CN112163517A (zh)
LU (1) LU500649B1 (zh)
WO (1) WO2022062242A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758260A (zh) * 2022-06-15 2022-07-15 成都鹏业软件股份有限公司 工地安全防护网检测方法及系统
CN117309900A (zh) * 2023-09-25 2023-12-29 中国水产科学研究院南海水产研究所 一种浅海渔网破损检测装置及控制方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116228757B (zh) * 2023-05-08 2023-08-29 山东省海洋科学研究院(青岛国家海洋科学研究中心) 一种基于图像处理算法的深海网箱网衣检测方法
CN117115688A (zh) * 2023-08-17 2023-11-24 广东海洋大学 基于深度学习的低亮度环境下死鱼识别计数系统及方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108734117A (zh) * 2018-05-09 2018-11-02 国网浙江省电力有限公司电力科学研究院 基于yolo的电缆设备外部腐蚀破损识别方法
CN109886344A (zh) * 2019-02-26 2019-06-14 广东工业大学 基于深度学习的皮革破损识别方法、系统及设备和介质
CN110163798A (zh) * 2019-04-18 2019-08-23 中国农业大学 渔场围网破损检测方法及系统
CN110223293A (zh) * 2019-06-21 2019-09-10 中国神华能源股份有限公司 列车车体破损的智能识别方法及识别装置
CN110335245A (zh) * 2019-05-21 2019-10-15 青岛科技大学 基于单目时空连续图像的网箱网衣破损监测方法及系统
KR20200067743A (ko) * 2018-11-02 2020-06-12 광주과학기술원 수중드론을 이용하는 어망감시장치, 및 그 장치의 제어방법
CN111583197A (zh) * 2020-04-23 2020-08-25 浙江大学 结合SSD及Resnet50网络的电力箱图片锈蚀破损识别方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109409365A (zh) * 2018-10-25 2019-03-01 江苏德劭信息科技有限公司 一种基于深度目标检测的待采摘水果识别和定位方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108734117A (zh) * 2018-05-09 2018-11-02 国网浙江省电力有限公司电力科学研究院 基于yolo的电缆设备外部腐蚀破损识别方法
KR20200067743A (ko) * 2018-11-02 2020-06-12 광주과학기술원 수중드론을 이용하는 어망감시장치, 및 그 장치의 제어방법
CN109886344A (zh) * 2019-02-26 2019-06-14 广东工业大学 基于深度学习的皮革破损识别方法、系统及设备和介质
CN110163798A (zh) * 2019-04-18 2019-08-23 中国农业大学 渔场围网破损检测方法及系统
CN110335245A (zh) * 2019-05-21 2019-10-15 青岛科技大学 基于单目时空连续图像的网箱网衣破损监测方法及系统
CN110223293A (zh) * 2019-06-21 2019-09-10 中国神华能源股份有限公司 列车车体破损的智能识别方法及识别装置
CN111583197A (zh) * 2020-04-23 2020-08-25 浙江大学 结合SSD及Resnet50网络的电力箱图片锈蚀破损识别方法

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114758260A (zh) * 2022-06-15 2022-07-15 成都鹏业软件股份有限公司 工地安全防护网检测方法及系统
CN114758260B (zh) * 2022-06-15 2022-10-18 成都鹏业软件股份有限公司 工地安全防护网检测方法及系统
CN117309900A (zh) * 2023-09-25 2023-12-29 中国水产科学研究院南海水产研究所 一种浅海渔网破损检测装置及控制方法
CN117309900B (zh) * 2023-09-25 2024-03-22 中国水产科学研究院南海水产研究所 一种浅海渔网破损检测装置及控制方法

Also Published As

Publication number Publication date
CN112163517A (zh) 2021-01-01
LU500649A1 (de) 2022-03-28
LU500649B1 (de) 2022-04-08

Similar Documents

Publication Publication Date Title
WO2022062242A1 (zh) 一种基于深度学习的水下成像鱼网破损识别方法及系统
Li et al. An effective data augmentation strategy for CNN-based pest localization and recognition in the field
CN111178197B (zh) 基于Mask R-CNN和Soft-NMS融合的群养粘连猪实例分割方法
Aslam et al. On the application of automated machine vision for leather defect inspection and grading: a survey
Zhu et al. Hierarchical convolutional neural network with feature preservation and autotuned thresholding for crack detection
Bhagat et al. WheatNet-lite: A novel light weight network for wheat head detection
CN112270681B (zh) 一种黄板害虫深度检测与计数方法与系统
CN113449806A (zh) 基于层次结构的二阶段林业害虫识别与检测系统及方法
CN109829484B (zh) 一种服饰分类方法、设备及计算机可读存储介质
Dulal et al. Automatic Cattle Identification using YOLOv5 and Mosaic Augmentation: A Comparative Analysis
Wu et al. A method for identifying grape stems using keypoints
Rajamohanan et al. An Optimized YOLO v5 Model for Tomato Leaf Disease Classification with Field Dataset
CN114022688A (zh) 一种实时的牲畜身份识别方法
Nath et al. Deep learning models for content-based retrieval of construction visual data
Nickolas Deep learning based betelvine leaf disease detection (piper betlel.)
Su et al. AIoT-cloud-integrated smart livestock surveillance via assembling deep networks with considering robustness and semantics availability
Raj et al. Steel Strip Quality Assurance With YOLOV7-CSF: A Coordinate Attention and SIoU Fusion Approach
Zhang et al. An approach for goose egg recognition for robot picking based on deep learning
Smink et al. Computer Vision on the Edge: Individual Cattle Identification in Real-Time With ReadMyCow System
CN116681961A (zh) 基于半监督方法和噪声处理的弱监督目标检测方法
Zhou et al. Unstructured road extraction and roadside fruit recognition in grape orchards based on a synchronous detection algorithm
Bello et al. Mask YOLOv7-Based Drone Vision System for Automated Cattle Detection and Counting
CN113221929A (zh) 一种图像处理方法以及相关设备
CN111079617A (zh) 家禽识别方法、装置、可读存储介质及电子设备
CN112308106A (zh) 一种图像标注的方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20955084

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20955084

Country of ref document: EP

Kind code of ref document: A1