WO2024098681A1 - 基于yolo自动识别的水电站平板闸门启闭方法 - Google Patents

基于yolo自动识别的水电站平板闸门启闭方法 Download PDF

Info

Publication number
WO2024098681A1
WO2024098681A1 PCT/CN2023/091629 CN2023091629W WO2024098681A1 WO 2024098681 A1 WO2024098681 A1 WO 2024098681A1 CN 2023091629 W CN2023091629 W CN 2023091629W WO 2024098681 A1 WO2024098681 A1 WO 2024098681A1
Authority
WO
WIPO (PCT)
Prior art keywords
gate
image
target
yolo
number plate
Prior art date
Application number
PCT/CN2023/091629
Other languages
English (en)
French (fr)
Inventor
张海龙
杜云华
陈钢
曾辉
刘稳
杨进
文宇
Original Assignee
中国长江电力股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国长江电力股份有限公司 filed Critical 中国长江电力股份有限公司
Publication of WO2024098681A1 publication Critical patent/WO2024098681A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field

Definitions

  • the invention belongs to the technical field of opening and closing of hydropower station gates, and relates to a method for opening and closing a flat gate of a hydropower station based on YOLO automatic recognition.
  • the gate of a hydropower station is a water-retaining metal structure. Its basic structure is mostly composed of flat panels, frames, supporting walking parts, hoists and water-stop components, and it moves vertically up and down in the gate slot.
  • the types of gantry cranes are divided into unit water inlet working doors, unit water inlet inspection doors, unit tailwater working doors, unit tailwater inspection doors, etc. according to their installation locations or functions.
  • Different hydropower stations may have dozens or hundreds of gate sections depending on the number of units and installed capacity.
  • the characteristics of hydropower station gates are fixed form, large size, and large number. There are general gates and special gates. Therefore, each section has a unique number to facilitate the operation and management of the hydropower station.
  • the technical problem to be solved by the present invention is to provide a method for opening and closing a flat gate of a hydropower station based on YOLO automatic recognition.
  • the method includes training a single-section gate image, establishing a training set, receiving the main hook encoder data of a gantry crane, judging the main lifting state, receiving the image of a camera on the gantry crane, performing preprocessing, matching the preprocessed image with the training set data, calculating the eigenvalue conformity, continuously observing the target and iterating, combining the main lifting state and the lifting weight to judge whether to lift into or out of the gate, updating the database in real time, and monitoring the video image of the gantry crane opening and closing the gate in real time based on the neural network architecture of YOLO.
  • the gate model and number are recognized and automatically recorded in real time, so that the intelligent gantry crane of the hydropower station can operate autonomously.
  • the technical solution adopted by the present invention is: a method for opening and closing a flat gate of a hydropower station based on YOLO automatic recognition, which comprises the following steps:
  • Step 1 train a single-section gate image to establish a training set; use a multi-layer neural network composed of YOLO to train, detect, and correct to form a training set for comparison;
  • Step 2 receiving the main hook encoder data and determining the main lifting status
  • Step 3 receiving the image from the camera and performing preprocessing
  • Step 4 matching the preprocessed image with the training set data and calculating the conformity of the feature values
  • Step 5 Continue to observe the target, iterate steps 3 and 4, and determine whether it is hoisted into the gate based on the main lifting status and lifting weight. The door is still hoisted out of the gate and the database is updated in real time.
  • step 1 In step 1,
  • Step 1-1 locate the gate, use image annotation software to process the collected image samples containing the gate of the hydropower station, generate a gate image sample data set, use the image data set to train a deep learning-based target detection model, and use the trained target detection model to locate the gate target in the input video frame image, obtain the bounding box prediction parameters of the gate target, and based on the bounding box prediction parameters of the gate target, crop the gate area from the video frame image to obtain an RGB color image of the gate area, that is, the gate image;
  • Step 1-2 locate the gate number plate and perform tilt correction on the gate image; use image annotation software to process the collected image samples containing the gate number plate of the hydropower station, generate a gate number plate image sample data set, use the image data set to train a deep learning-based target detection model, and use the trained target detection model to locate the gate number plate target in the input video frame image, obtain the bounding box prediction parameters of the gate number plate target, and based on the bounding box prediction parameters of the gate number plate target, crop the gate number plate area from the video frame image to obtain an RGB color image of the gate number plate area, that is, the gate number plate image;
  • Step 1-3 identify the gate number, perform preprocessing such as binarization on the corrected gate number plate image, and use the trained target detection model to identify all characters in the gate number plate image and record them.
  • step 2
  • Step 2-1 establish the gantry crane coordinate system, the trolley movement direction is the x-axis, the trolley movement direction is the y-axis, and the main lifting movement direction is the z-axis;
  • Step 2-2 measure the coordinate value of the gate slot in the gate machine coordinate system, establish a mapping relationship between the gate slot vector group and the coordinate value, and the gate slot corresponds to Location (x, y);
  • Step 2-3 determine whether the gate is in an ascending or descending state according to the change in the movement direction of the main lifting hook in the z-axis.
  • Step 3-1 installing a camera at an appropriate position of the gantry crane
  • Step 3-2 when the movement of the gantry crane is detected, the camera takes real-time images of the hoisting area;
  • Step 3-3 use the projection transformation algorithm to perform image geometric correction on the image, and use the histogram equalization algorithm to correct the image grayscale value, so that it meets the requirements of establishing a unified calculation framework before operator extraction;
  • Step 3-4 use FFT and wavelet transform to reduce the noise of the image.
  • step 4
  • Step 4-1 For the preprocessed image, use the BackBone reasoning strategy to detect the matching degree between the target image and the training result set, and provide an extraction result with high matching degree;
  • Step 4-2 for images that meet expectations, extract the texture features of the image blocks to form a Mosaic comparison benchmark, and compare them in the video frames, merging the targets with consistent texture features into the temporal motion of the same target.
  • step 5
  • Step 5-1 Use the Ethernet-based IEEE1588 clock synchronization protocol to ensure the time synchronization of sensors, image recognition and other modules;
  • Step 5-2 If the main hoist is detected to be moving upward and a gate image is detected at the same time, it is determined that a gate is hoisted out, the real-time position of the gate crane is obtained through the trolley positioning module, and the gate information in the corresponding gate slot is changed;
  • Step 5-3 if the main hoist is detected to move up and down, and a gate image is detected at the same time, it is determined that a gate is hoisted in, and the real-time position of the gate crane is obtained through the trolley positioning module, and the gate information in the corresponding gate slot is changed.
  • the YOLO-based neural network architecture monitors the video images of the gantry crane opening and closing gates in real time. Combined with information such as the gantry crane position, main hook status, and lifting load, it can autonomously identify the gate model and number in real time and automatically record them.
  • FIG. 1 is a flow chart of the present invention.
  • FIG. 2 is a diagram showing the gate opening and closing states of the present invention.
  • FIG. 3 is a motion direction diagram of the coordinate system constructed by the gantry crane of the present invention.
  • FIG. 4 is a schematic diagram of the intelligent control system of the present invention.
  • a method for opening and closing a flat gate of a hydropower station based on YOLO automatic recognition includes the following steps:
  • Step 1 train a single-section gate image to establish a training set; use a multi-layer neural network composed of YOLO to train, detect, and correct to form a training set for comparison;
  • Step 2 receiving the main hook encoder data and determining the main lifting status
  • Step 3 receiving the image from the camera and performing preprocessing
  • Step 4 matching the preprocessed image with the training set data and calculating the conformity of the feature values
  • Step 5 continuously observe the target, iterate steps 3 and 4, combine the main lifting status and lifting weight to determine whether to lift into or out of the gate, and update the database in real time.
  • YOLO is the abbreviation of You Only Look Once, which is an object recognition and positioning algorithm based on deep neural network. Its biggest feature is its fast running speed.
  • the detection rate of YOLOV1 can reach 45 frames per second, which can be used in real-time systems.
  • the gate is characterized by its large size, with length and width in the order of "meters", and the speed of the gate machine to open and close the gate is about 2 meters per second, which just coincides with the large and fast application target scenarios of YOLO.
  • a YOLO-based neural network architecture is set up to monitor the video images of the gantry crane opening and closing the gate in real time. Combined with information such as the gantry crane position, main hook status, and lifting load, the gate model and number can be autonomously identified in real time and automatically recorded.
  • step 1 in step 1,
  • Step 1-1 locate the gate, use image annotation software to process the collected image samples containing the gate of the hydropower station, generate a gate image sample data set, use the image data set to train a deep learning-based target detection model, and use the trained target detection model to locate the gate target in the input video frame image, obtain the bounding box prediction parameters of the gate target, and based on the bounding box prediction parameters of the gate target, crop the gate area from the video frame image to obtain an RGB color image of the gate area, that is, the gate image;
  • Step 1-2 locate the gate number plate and perform tilt correction on the gate image; use image annotation software to process the collected image samples containing the gate number plate of the hydropower station, generate a gate number plate image sample data set, use the image data set to train a deep learning-based target detection model, and use the trained target detection model to locate the gate number plate target in the input video frame image, obtain the bounding box prediction parameters of the gate number plate target, and based on the bounding box prediction parameters of the gate number plate target, crop the gate number plate area from the video frame image to obtain an RGB color image of the gate number plate area, that is, the gate number plate image;
  • Step 1-3 identify the gate number, perform preprocessing such as binarization on the corrected gate number plate image, and use the trained target detection model to identify all characters in the gate number plate image and record them.
  • step 2 In a preferred embodiment, in step 2,
  • Step 2-1 establish the gantry crane coordinate system, the trolley movement direction is the x-axis, the trolley movement direction is the y-axis, and the main lifting movement direction is the z-axis;
  • Step 2-2 measure the coordinate value of the gate slot in the gate machine coordinate system, establish a mapping relationship between the gate slot vector group and the coordinate value, and the gate slot corresponds to Location (x, y);
  • Step 2-3 determine whether the gate is in an ascending or descending state according to the change in the movement direction of the main lifting hook in the z-axis.
  • step 3 In a preferred embodiment, in step 3,
  • Step 3-1 installing a camera at an appropriate position of the gantry crane
  • Step 3-2 when the movement of the gantry crane is detected, the camera takes real-time images of the hoisting area;
  • Step 3-3 use the projection transformation algorithm to perform image geometric correction on the image, and use the histogram equalization algorithm to correct the image grayscale value, so that it meets the requirements of establishing a unified calculation framework before operator extraction;
  • Step 3-4 use FFT and wavelet transform to reduce the noise of the image.
  • step 4 In a preferred embodiment, in step 4,
  • Step 4-1 For the preprocessed image, use the BackBone reasoning strategy to detect the matching degree between the target image and the training result set, and provide an extraction result with high matching degree;
  • Step 4-2 for images that meet expectations, extract the texture features of the image blocks to form a Mosaic comparison benchmark, and compare them in the video frames, merging the targets with consistent texture features into the temporal motion of the same target.
  • step 5 in step 5,
  • Step 5-1 Use the Ethernet-based IEEE1588 clock synchronization protocol to ensure the time synchronization of sensors, image recognition and other modules;
  • Step 5-2 If the main hoist is detected to be moving upward and a gate image is detected at the same time, it is determined that a gate is hoisted out, the real-time position of the gate crane is obtained through the trolley positioning module, and the gate information in the corresponding gate slot is changed;
  • Step 5-3 If the main hoist is detected to be moving downward and a gate image is detected at the same time, it is determined that a gate is hoisted in, and the real-time position of the gate crane is obtained through the trolley positioning module, and the gate information in the corresponding gate slot is changed;
  • the door machine system of the intelligent control system mainly includes a main lifting frequency converter, a trolley frequency converter and a trolley frequency converter.
  • the main lifting frequency converter is electrically connected to the lifting encoder
  • the trolley frequency converter is electrically connected to the trolley positioning module
  • the trolley frequency converter is connected to the trolley positioning module.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computing Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

一种基于YOLO自动识别的水电站平板闸门启闭方法，通过训练单节闸门图像，建立训练集，接收门式起重机主钩编码器数据，判断主起升状态，接收门式起重机上摄像机的图像，进行预处理，根据已预处理的图片与训练集数据进行匹配，计算特征值符合度，持续观测目标并迭代，结合主起升状态、起吊重量，判断是吊入闸门还是吊出闸门，实时更新数据库，基于YOLO的神经网络架构，实时监测门式起重机启闭闸门的视频图像，结合门机位置、主钩状态以及起吊载重量信息，实时自主识别闸门型号、编号，并自动记录，使水电站智能门机自主作业。

Description

基于YOLO自动识别的水电站平板闸门启闭方法 技术领域
本发明属于水电站闸门启闭技术领域，涉及一种基于YOLO自动识别的水电站平板闸门启闭方法。
背景技术
水电站闸门是挡水金结设备，基本构建多由平直面板、构架、支撑行走部件、吊具和止水等部件组成，在门槽内直升直降运动；门机种类根据安装位置或功能的不同，分为机组进水口工作门、机组进水口检修门、机组尾水工作门、机组尾水检修门等；不同水电站，根据机组台数、装机容量不同，可能会有几十或几百节闸门，水电站闸门的特点是形式确定、尺寸大、数量多，有通用闸门和专用闸门，因此，每一节都有唯一编号，以便于水电站运行管理。
目前，在水电站闸门启闭或者位置异动后，由现场作业人员观察并记录闸门位置变化情况，不能实现智能门机自主作业。
发明内容
本发明所要解决的技术问题是提供一种基于YOLO自动识别的水电站平板闸门启闭方法，通过训练单节闸门图像，建立训练集，接收门式起重机主钩编码器数据，判断主起升状态，接收门式起重机上摄像机的图像，进行预处理，根据已预处理的图片与训练集数据进行匹配，计算特征值符合度，持续观测目标并迭代，结合主起升状态、起吊重量，判断是吊入闸门还是吊出闸门，实时更新数据库，基于YOLO的神经网络架构，实时监测门式起重机启闭闸门的视频图像，结合门机位置、主钩状态以及起吊载重量信息，实时自主识别闸门型号、编号，并自动记录，使水电站智能门机自主作业。
为解决上述技术问题，本发明所采用的技术方案是：一种基于YOLO自动识别的水电站平板闸门启闭方法，它包括如下步骤：
步骤1，训练单节闸门图像，建立训练集；利用由YOLO构成的多层神经网络进行训练，检测，矫正，形成可供比对的训练集；
步骤2，接收主钩编码器数据，判断主起升状态；
步骤3，接收摄像机的图像，进行预处理；
步骤4，根据已预处理的图片与训练集数据进行匹配，计算特征值符合度；
步骤5，持续观测目标，迭代步骤3、步骤4，结合主起升状态、起吊重量，判断是吊入闸 门还是吊出闸门，实时更新数据库。
在步骤1中，
步骤1-1，定位闸门，利用图像标注软件对收集到的包含水电站闸门的图像样本进行处理，生成闸门图像样本数据集，利用图像数据集训练基于深度学习的目标检测模型，并利用训练好的目标检测模型对输入的视频帧图像中的闸门目标进行定位，获取闸门目标的边界框预测参数，基于闸门目标的边界框预测参数，从视频帧图像中裁剪出闸门区域，得到闸门区域的RGB彩色图像，即闸门图像；
步骤1-2，定位闸门号牌，对闸门图像进行倾斜校正；利用图像标注软件对收集到的包含水电站闸门编号牌的图像样本进行处理，生成闸门编号牌图像样本数据集，利用图像数据集训练基于深度学习的目标检测模型，并利用训练好的目标检测模型对输入的视频帧图像中的闸门编号牌目标进行定位，获取闸门编号牌目标的边界框预测参数，基于闸门编号牌目标的边界框预测参数，从视频帧图像中裁剪出闸门编号牌区域，得到闸门编号牌区域的RGB彩色图像，即闸门编号牌图像；
步骤1-3，识别闸门编号，对校正后的闸门编号牌图像进行二值化等预处理，利用训练好的目标检测模型识别闸门编号牌图像中的所有字符并记录。
在步骤2中，
步骤2-1，建立门式起重机坐标系，大车运动方向为x轴，小车运动方向为y轴，主起升运动方向为z轴；
步骤2-2，测量闸门门槽在门机坐标系内的坐标值，建立门槽向量组与坐标值的映射关系，门槽对应于Location(x，y)；
步骤2-3，根据主起升吊钩在z轴运动方向的变化确定闸门是处于上升还是下降状态。
在步骤3中，
步骤3-1，在门式起重机适当位置装设摄像机；
步骤3-2，当检测到门式起重机动作后，摄像机实时拍摄吊装区域图像；
步骤3-3，利用投影变换算法对图像进行图像几何矫正、利用直方图均衡化算法对图像灰度值进行矫正，使之符合算子提取前，建立统一计算框架的要求；
步骤3-4，利用FFT、小波变换对图像进行降噪处理。
在步骤4中，
步骤4-1，对预处理后的图像，利用BackBone推理策略，检测目标图像和训练结果集的匹配程度，并提供高符合度的提取结果；
步骤4-2，对于符合预期的图像，提取图像区块的纹理特征，形成Mosaic对比基准，并在视频帧中予以对比，将纹理特征符合的目标合并为同一目标的时序运动。
在步骤5中，
步骤5-1，采用基于以太网的IEEE1588时钟同步协议，确保传感器及图像识别等模块时间同步；
步骤5-2，如果检测到主起升向上运动，同时检测到有闸门图像，判断有闸门吊出，通过大小车定位模块得到门机实时位置，更改相应门槽内闸门信息；
步骤5-3，如果检测到主起升向上下运动，同时检测到有闸门图像，判断有闸门吊入，通过大小车定位模块得到门机实时位置，更改相应门槽内闸门信息。
本发明的主要有益效果在于：
基于YOLO的神经网络架构，实时监测门式起重机启闭闸门的视频图像，结合门机位置、主钩状态以及起吊载重量等信息，能够实时自主识别闸门型号、编号，并自动记录。
实现了水电站智能门机自主作业。
结合主起升状态、起吊重量，判断是吊入闸门还是吊出闸门。
综合利用图像处理技术和深度学习技术实现了水电站闸门的实时动态精准识别，为水电站金结设备数字化管理奠定了技术基础。
附图说明
下面结合附图和实施例对本发明作进一步说明：
图1为本发明的流程图。
图2为本发明闸门启闭状态图。
图3为本发明门式起重机构建的坐标系运动方向图。
图4为本发明智能控制系统简图。
具体实施方式
如图1～图4中，一种基于YOLO自动识别的水电站平板闸门启闭方法，它包括如下步骤：
步骤1，训练单节闸门图像，建立训练集；利用由YOLO构成的多层神经网络进行训练，检测，矫正，形成可供比对的训练集；
步骤2，接收主钩编码器数据，判断主起升状态；
步骤3，接收摄像机的图像，进行预处理；
步骤4，根据已预处理的图片与训练集数据进行匹配，计算特征值符合度；
步骤5，持续观测目标，迭代步骤3、步骤4，结合主起升状态、起吊重量，判断是吊入闸门还是吊出闸门，实时更新数据库。
优选地，YOLO即是YouOnlyLookOnce的缩写，是一种基于深度神经网络的对象识别和定位算法，其最大的特点是运行速度快，YOLOV1的检测速率可以达到45帧/秒，可以用于实时系统；闸门的特点是尺寸大，长宽尺寸在“米”数量级，门机启闭闸门的速度在2米/秒左右，刚好与足YOLO大目标、快速的应用目标场景相吻合。
优选地，搭设基于YOLO的神经网络架构，实时监测门式起重机启闭闸门的视频图像，结合门机位置、主钩状态以及起吊载重量等信息，能够实时自主识别闸门型号、编号，并自动记录。
优选的方案中，在步骤1中，
步骤1-1，定位闸门，利用图像标注软件对收集到的包含水电站闸门的图像样本进行处理，生成闸门图像样本数据集，利用图像数据集训练基于深度学习的目标检测模型，并利用训练好的目标检测模型对输入的视频帧图像中的闸门目标进行定位，获取闸门目标的边界框预测参数，基于闸门目标的边界框预测参数，从视频帧图像中裁剪出闸门区域，得到闸门区域的RGB彩色图像，即闸门图像；
步骤1-2，定位闸门号牌，对闸门图像进行倾斜校正；利用图像标注软件对收集到的包含水电站闸门编号牌的图像样本进行处理，生成闸门编号牌图像样本数据集，利用图像数据集训练基于深度学习的目标检测模型，并利用训练好的目标检测模型对输入的视频帧图像中的闸门编号牌目标进行定位，获取闸门编号牌目标的边界框预测参数，基于闸门编号牌目标的边界框预测参数，从视频帧图像中裁剪出闸门编号牌区域，得到闸门编号牌区域的RGB彩色图像，即闸门编号牌图像；
步骤1-3，识别闸门编号，对校正后的闸门编号牌图像进行二值化等预处理，利用训练好的目标检测模型识别闸门编号牌图像中的所有字符并记录。
优选的方案中，在步骤2中，
步骤2-1，建立门式起重机坐标系，大车运动方向为x轴，小车运动方向为y轴，主起升运动方向为z轴；
步骤2-2，测量闸门门槽在门机坐标系内的坐标值，建立门槽向量组与坐标值的映射关系，门槽对应于Location(x，y)；
步骤2-3，根据主起升吊钩在z轴运动方向的变化确定闸门是处于上升还是下降状态。
优选的方案中，在步骤3中，
步骤3-1，在门式起重机适当位置装设摄像机；
步骤3-2，当检测到门式起重机动作后，摄像机实时拍摄吊装区域图像；
步骤3-3，利用投影变换算法对图像进行图像几何矫正、利用直方图均衡化算法对图像灰度值进行矫正，使之符合算子提取前，建立统一计算框架的要求；
步骤3-4，利用FFT、小波变换对图像进行降噪处理。
优选的方案中，在步骤4中，
步骤4-1，对预处理后的图像，利用BackBone推理策略，检测目标图像和训练结果集的匹配程度，并提供高符合度的提取结果；
步骤4-2，对于符合预期的图像，提取图像区块的纹理特征，形成Mosaic对比基准，并在视频帧中予以对比，将纹理特征符合的目标合并为同一目标的时序运动。
优选的方案中，在步骤5中，
步骤5-1，采用基于以太网的IEEE1588时钟同步协议，确保传感器及图像识别等模块时间同步；
步骤5-2，如果检测到主起升向上运动，同时检测到有闸门图像，判断有闸门吊出，通过大小车定位模块得到门机实时位置，更改相应门槽内闸门信息；
步骤5-3，如果检测到主起升向下运动，同时检测到有闸门图像，判断有闸门吊入，通过大小车定位模块得到门机实时位置，更改相应门槽内闸门信息；
优选地，智能控制系统的门机系统主要包括主起升变频器、大车变频器和小车变频器，主起升变频器与起升编码器电性连接，大车变频器与大车定位模块电性连接，小车变频器与小车定位模块连接。
上述的实施例仅为本发明的优选技术方案，而不应视为对于本发明的限制，本申请中的实施例及实施例中的特征在不冲突的情况下，可以相互任意组合。本发明的保护范围应以权利要求记载的技术方案，包括权利要求记载的技术方案中技术特征的等同替换方案为保护范围。即在此范围内的等同替换改进，也在本发明的保护范围之内。

Claims (6)

  1. 一种基于YOLO自动识别的水电站平板闸门启闭方法，其特征是，它包括如下步骤：
    步骤1，训练单节闸门图像，建立训练集；利用由YOLO构成的多层神经网络进行训练，检测，矫正，形成可供比对的训练集；
    步骤2，接收主钩编码器数据，判断主起升状态；
    步骤3，接收摄像机的图像，进行预处理；
    步骤4，根据已预处理的图片与训练集数据进行匹配，计算特征值符合度；
    步骤5，持续观测目标，迭代步骤3、步骤4，结合主起升状态、起吊重量，判断是吊入闸门还是吊出闸门，实时更新数据库。
  2. 根据权利要求1所述的基于YOLO自动识别的水电站平板闸门启闭方法，其特征是，在步骤1中，
    步骤1-1，定位闸门，利用图像标注软件对收集到的包含水电站闸门的图像样本进行处理，生成闸门图像样本数据集，利用图像数据集训练基于深度学习的目标检测模型，并利用训练好的目标检测模型对输入的视频帧图像中的闸门目标进行定位，获取闸门目标的边界框预测参数，基于闸门目标的边界框预测参数，从视频帧图像中裁剪出闸门区域，得到闸门区域的RGB彩色图像，即闸门图像；
    步骤1-2，定位闸门号牌，对闸门图像进行倾斜校正；利用图像标注软件对收集到的包含水电站闸门编号牌的图像样本进行处理，生成闸门编号牌图像样本数据集，利用图像数据集训练基于深度学习的目标检测模型，并利用训练好的目标检测模型对输入的视频帧图像中的闸门编号牌目标进行定位，获取闸门编号牌目标的边界框预测参数，基于闸门编号牌目标的边界框预测参数，从视频帧图像中裁剪出闸门编号牌区域，得到闸门编号牌区域的RGB彩色图像，即闸门编号牌图像；
    步骤1-3，识别闸门编号，对校正后的闸门编号牌图像进行二值化等预处理，利用训练好的目标检测模型识别闸门编号牌图像中的所有字符并记录。
  3. 根据权利要求1所述的基于YOLO自动识别的水电站平板闸门启闭方法，其特征是，在步骤2中，
    步骤2-1，建立门式起重机坐标系，大车运动方向为x轴，小车运动方向为y轴，主起升运动方向为z轴；
    步骤2-2，测量闸门门槽在门机坐标系内的坐标值，建立门槽向量组与坐标值的映射关系，门槽对应于Location(x，y)；
    步骤2-3，根据主起升吊钩在z轴运动方向的变化确定闸门是处于上升还是下降状态。
  4. 根据权利要求1所述的基于YOLO自动识别的水电站平板闸门启闭方法，其特征是，在步骤3中，
    步骤3-1，在门式起重机适当位置装设摄像机；
    步骤3-2，当检测到门式起重机动作后，摄像机实时拍摄吊装区域图像；
    步骤3-3，利用投影变换算法对图像进行图像几何矫正、利用直方图均衡化算法对图像灰度值进行矫正，使之符合算子提取前，建立统一计算框架的要求；
    步骤3-4，利用FFT、小波变换对图像进行降噪处理。
  5. 根据权利要求1所述的基于YOLO自动识别的水电站平板闸门启闭方法，其特征是，在步骤4中，
    步骤4-1，对预处理后的图像，利用BackBone推理策略，检测目标图像和训练结果集的匹配程度，并提供高符合度的提取结果；
    步骤4-2，对于符合预期的图像，提取图像区块的纹理特征，形成Mosaic对比基准，并在视频帧中予以对比，将纹理特征符合的目标合并为同一目标的时序运动。
  6. 根据权利要求1所述的基于YOLO自动识别的水电站平板闸门启闭方法，其特征是，在步骤5中，
    步骤5-1，采用基于以太网的IEEE1588时钟同步协议，确保传感器及图像识别等模块时间同步；
    步骤5-2，如果检测到主起升向上运动，同时检测到有闸门图像，判断有闸门吊出，通过大小车定位模块得到门机实时位置，更改相应门槽内闸门信息；
    步骤5-3，如果检测到主起升向上下运动，同时检测到有闸门图像，判断有闸门吊入，通过大小车定位模块得到门机实时位置，更改相应门槽内闸门信息。
PCT/CN2023/091629 2022-11-08 2023-04-28 基于yolo自动识别的水电站平板闸门启闭方法 WO2024098681A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211393295.4 2022-11-08
CN202211393295.4A CN115761316B (zh) 2022-11-08 2022-11-08 基于yolo自动识别的水电站平板闸门启闭方法

Publications (1)

Publication Number Publication Date
WO2024098681A1 true WO2024098681A1 (zh) 2024-05-16

Family

ID=85368176

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/091629 WO2024098681A1 (zh) 2022-11-08 2023-04-28 基于yolo自动识别的水电站平板闸门启闭方法

Country Status (2)

Country Link
CN (1) CN115761316B (zh)
WO (1) WO2024098681A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118314527A (zh) * 2024-06-07 2024-07-09 浙江大华技术股份有限公司 起吊安全监测方法、装置、终端及计算机可读存储介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115761316B (zh) * 2022-11-08 2024-01-05 中国长江电力股份有限公司 基于yolo自动识别的水电站平板闸门启闭方法
CN116579230A (zh) * 2023-03-31 2023-08-11 中国长江电力股份有限公司 基于tcn神经网络的闸门启闭状态实时预警方法及装置
CN116819974B (zh) * 2023-08-30 2023-11-03 北京建筑大学 基于深度强化学习的排水管网末端智能排水方法及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107844067A (zh) * 2017-12-07 2018-03-27 国家电网公司 一种水电站闸门在线状态监测控制方法及监测系统
KR102159620B1 (ko) * 2020-05-06 2020-09-24 (주)동천기공 Ai 및 딥러닝 기반의 스마트 홍수관리 수문제어 자동화 시스템
CN112001320A (zh) * 2020-08-25 2020-11-27 天地伟业技术有限公司 一种基于视频的闸门检测方法
CN112906593A (zh) * 2021-03-03 2021-06-04 河海大学 一种基于Faster RCNN的水闸图像识别方法
CN113554123A (zh) * 2021-09-18 2021-10-26 江苏禹治流域管理技术研究院有限公司 一种基于声光联动的采砂船自动识别方法
CN115761316A (zh) * 2022-11-08 2023-03-07 中国长江电力股份有限公司 基于yolo自动识别的水电站平板闸门启闭方法

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3004501B2 (ja) * 1993-08-02 2000-01-31 三菱重工業株式会社 車種判別装置
CN110458161B (zh) * 2019-07-15 2023-04-18 天津大学 一种结合深度学习的移动机器人门牌定位方法
CN111382680B (zh) * 2020-02-26 2023-04-25 浙江大华技术股份有限公司 一种闸门监测方法、终端及计算机可读存储介质
CN113391607A (zh) * 2021-06-22 2021-09-14 四川华能太平驿水电有限责任公司 一种基于深度学习的水电站闸门控制方法及系统
CN113820976B (zh) * 2021-08-30 2023-09-29 长江勘测规划设计研究有限责任公司 基于人工智能的闸门智能控制方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107844067A (zh) * 2017-12-07 2018-03-27 国家电网公司 一种水电站闸门在线状态监测控制方法及监测系统
KR102159620B1 (ko) * 2020-05-06 2020-09-24 (주)동천기공 Ai 및 딥러닝 기반의 스마트 홍수관리 수문제어 자동화 시스템
CN112001320A (zh) * 2020-08-25 2020-11-27 天地伟业技术有限公司 一种基于视频的闸门检测方法
CN112906593A (zh) * 2021-03-03 2021-06-04 河海大学 一种基于Faster RCNN的水闸图像识别方法
CN113554123A (zh) * 2021-09-18 2021-10-26 江苏禹治流域管理技术研究院有限公司 一种基于声光联动的采砂船自动识别方法
CN115761316A (zh) * 2022-11-08 2023-03-07 中国长江电力股份有限公司 基于yolo自动识别的水电站平板闸门启闭方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118314527A (zh) * 2024-06-07 2024-07-09 浙江大华技术股份有限公司 起吊安全监测方法、装置、终端及计算机可读存储介质

Also Published As

Publication number Publication date
CN115761316B (zh) 2024-01-05
CN115761316A (zh) 2023-03-07

Similar Documents

Publication Publication Date Title
WO2024098681A1 (zh) 基于yolo自动识别的水电站平板闸门启闭方法
CN110070324A (zh) 一种集装箱码头智能理货系统
WO2019080367A1 (zh) 一种机械设备健康状态评估方法
CN106446869A (zh) 基于图像智能识别算法的隔离开关状态自动检测方法
CN112149522A (zh) 电缆通道智能可视化防外破监测系统及方法
CN112237183B (zh) 一种可监测的超声波闭环驱鸟装置及驱鸟方法
CN116624065B (zh) 一种智能门窗自动折叠调控方法
CN112017195A (zh) 一种应用于城市轨道交通的智能综合监控系统
CN113971781A (zh) 建筑结构施工进度的识别方法、装置、客户端和存储介质
CN118062754B (zh) 基于数据分析的吊塔运行风险监管预警系统
CN112001320B (zh) 一种基于视频的闸门检测方法
CN113269435B (zh) 一种新能源场站运行状态耦合监测评估系统
CN118013197A (zh) 一种基于深度学习算法的火炬流量监测应急系统及方法
CN116523893B (zh) 一种基于机器视觉的飞钢检测系统及方法
CN115862021A (zh) 基于yolo的水电站闸门自动识别方法
CN112801072B (zh) 一种基于计算机视觉的电梯非平层开门故障识别装置及方法
CN114332736A (zh) 一种电力场所消防安全风险分析方法及系统
CN105590329B (zh) 一种油气田视频监控中运动目标的智能检测算法
CN118534829B (zh) 基于物联网的液流储能设备全生命周期监控方法和系统
CN117804811B (zh) 注意力机制深度神经网络港机装备健康诊断系统及方法
CN114677745B (zh) 一种智能电缆井安全管理系统
CN113946154B (zh) 一种巡线机器人的视觉识别方法和系统
CN216144360U (zh) 一种基于物联网的生产现场实时监测系统
CN115689844B (zh) 一种基于多维引擎的智能数据管理平台、搭建方法
CN117912153A (zh) 一种应用智慧社区的自动化出入门禁管理系统和方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23887382

Country of ref document: EP

Kind code of ref document: A1