WO2024093287A1 - 一种基于改进u2型网络的仪表识别方法 - Google Patents

一种基于改进u2型网络的仪表识别方法 Download PDF

Info

Publication number
WO2024093287A1
WO2024093287A1 PCT/CN2023/103126 CN2023103126W WO2024093287A1 WO 2024093287 A1 WO2024093287 A1 WO 2024093287A1 CN 2023103126 W CN2023103126 W CN 2023103126W WO 2024093287 A1 WO2024093287 A1 WO 2024093287A1
Authority
WO
WIPO (PCT)
Prior art keywords
instrument
improved
image
pointer
scale
Prior art date
Application number
PCT/CN2023/103126
Other languages
English (en)
French (fr)
Inventor
张登银
李经纬
Original Assignee
南京邮电大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京邮电大学 filed Critical 南京邮电大学
Priority to US18/469,583 priority Critical patent/US20240005639A1/en
Publication of WO2024093287A1 publication Critical patent/WO2024093287A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting

Definitions

  • the invention relates to an instrument recognition method based on an improved U2 network, belonging to the technical field of computer vision image processing.
  • Pointer meters are the most commonly used instruments in industry. They have many advantages such as reliability, simple maintenance, and low price. They can also adapt to various scenarios, such as high pressure, high temperature, and high radiation environments. However, due to the harsh working environment, meter reading operations are usually not completed manually, so automatic reading has become a solution to this problem.
  • Image segmentation technology based on deep learning has stronger stability against external interference factors and can achieve more accurate automatic readings, thus ensuring real-time monitoring in the industrial production process.
  • This solution can save time, manpower and material resources without large-scale rectification of traditional workshops, and can also achieve relatively good automatic recognition and reading effects.
  • U 2 -Net is a two-layer nested U-shaped network structure, which mainly includes residual U-blocks (RSU) for extracting multi-scale features and an outer U-shaped structure connecting RSU.
  • RSU residual U-blocks
  • the algorithm does not need to use a pre-trained backbone network for image classification during the design process, and can extract feature maps with higher resolution at a low computational cost.
  • RSU consists of three parts: an input convolution layer for channel conversion of the input, a U-shaped structure for extracting context information at multi-scale, and a residual connection layer that connects the input layer and the middle layer through residual blocks.
  • the present invention adopts the following technical solutions:
  • a method for identifying an instrument based on an improved U2 type network comprising:
  • the scale array is subjected to noise reduction processing by using mean filtering to obtain the number of dial scales Point_num passed by the pointer and the total number of dial scales Nums;
  • the ring-shaped dial scale on the masked image is expanded into a two-dimensional matrix image.
  • X cx+r*sin ⁇
  • Y cy-r*cos ⁇
  • X and Y are the coordinates after unfolding into a rectangle
  • cx and cy are the center coordinates of the ring
  • r is the radius
  • ⁇ is the rotation angle corresponding to the scale.
  • the final meter reading is calculated, including:
  • the image segmentation model based on the improved U2 -type network includes a residual U-type block RSU for extracting multi-scale features and an outer U-type structure connecting the RSU; wherein the RSU is composed of three parts: an input convolution layer for performing channel conversion on the input, a U-type structure for extracting context information at multi-scale, and a residual connection layer for connecting the input layer and the intermediate layer through a residual block; wherein the convolution in the residual U-type block RSU adopts grouped convolution Grouped Conv; for grouped convolution Grouped Conv, the input feature map size is H*W*C 1 /g, so C 2 convolution kernels of size H*W*C 1 /g are required. Since the splicing operation does not require additional parameters, the parameter amount is H*W*C 1 /g, which is 1/g of the original.
  • the training method of the image segmentation model based on the improved U2 network includes:
  • Obtain a labeled instrument image dataset and divide the dataset into a training set, a validation set, and a test set, wherein the labels include a pointer, a dial scale, and a background;
  • the data set is used to train and test an image segmentation model based on an improved U2 network, thereby obtaining a trained image segmentation model based on an improved U2 network.
  • using the data set to train an image segmentation model based on an improved U2 network includes:
  • the jpg format instrument image is converted into a single-channel png format image, with 0 for background, 1 for pointer, and 2 for scale. Then the original image and the corresponding png image are input into the network for training, and then the obtained scale and pointer are compared with the corresponding png image. By continuously adjusting the parameters to minimize the gap between the two, a trained image segmentation model based on the improved U2 network is obtained.
  • the meter preset data includes the scale value, division value and range information of the same meter.
  • the present invention provides an instrument identification device based on an improved U2 type network, comprising a processor and a storage medium;
  • the storage medium is used to store instructions
  • the processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.
  • the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.
  • the present invention proposes an instrument recognition method based on an improved U2 network.
  • GC Grouped Conv
  • the network is used to segment the dial and the pointer;
  • the dial is expanded into a matrix image by traditional mathematical methods, and then the two-dimensional image is converted into a one-dimensional array;
  • the scale value array is subjected to noise reduction processing by using mean filtering;
  • the position of the pointer corresponding to the scale value is located by using the peak value, and the indication is output according to the scale value and preset data;
  • the present invention mainly uses the improved U2 network
  • the network is used to segment the pointer instrument, and then the obtained dial is automatically read through the traditional computer vision method;
  • the pointer instrument has a simple structure, is easy to install, and is insensitive to industrial factors such as electromagnetic interference.
  • this method has the advantages of high accuracy, high reliability, fast reading, and low cost, which can greatly improve work efficiency.
  • the scale and pointer in the pointer instrument image are obtained through image segmentation technology, and then the final reading is calculated according to the preset dial information, realizing unmanned real-time online reading, improving work efficiency, and saving manpower and material resources.
  • FIG1 is a schematic diagram of the Grouped Conv operation of an embodiment of the present invention.
  • FIG2 is a flow chart of a method provided by an embodiment of the present invention.
  • Figure 3 is the RSU module proposed by the U 2 -Net segmentation network
  • Figure 4 is a diagram of the U 2 -Net segmentation network structure
  • Figure 5 is the improved RSU module.
  • An instrument identification method based on an improved U2 type network comprises:
  • the scale array is subjected to noise reduction processing by using mean filtering to obtain the number of dial scales Point_num passed by the pointer and the total number of dial scales Nums;
  • X and Y are the coordinates after unfolding into a rectangle
  • cx and cy are the center coordinates of the ring
  • r is the radius
  • ⁇ is the rotation angle corresponding to the scale.
  • the final meter reading is calculated, including:
  • the image segmentation model based on the improved U2 -type network includes a residual U-type block RSU for extracting multi-scale features and an outer U-type structure connecting the RSU; wherein the RSU is composed of three parts: an input convolution layer for performing channel conversion on the input, a U-type structure for extracting context information at multi-scale, and a residual connection layer for connecting the input layer and the intermediate layer through a residual block; wherein the convolution in the residual U-type block RSU adopts grouped convolution Grouped Conv; for grouped convolution Grouped Conv, the input feature map size is H*W*C 1 /g, so C 2 convolution kernels of H*W*C 1 /g size are required. Since the splicing operation does not require additional parameters, the parameter amount is H*W*C 1 /g, which is 1/g of the original.
  • the training method of the image segmentation model based on the improved U2 network includes:
  • Obtain a labeled instrument image dataset and divide the dataset into a training set, a validation set, and a test set, wherein the labels include a pointer, a dial scale, and a background;
  • the data set is used to train and test an image segmentation model based on an improved U2 network, thereby obtaining a trained image segmentation model based on an improved U2 network.
  • using the data set to train an image segmentation model based on an improved U2 network includes:
  • the meter preset data includes the scale value, division value and range information of the same meter.
  • an instrument identification method based on an improved U 2 type network includes the following steps:
  • Step 1 Obtain the same pointer instrument image dataset obtained by real shooting, and divide this dataset into training set, validation set and test set in a ratio of 6:2:2. At the same time, record the fixed information such as the scale value, division value and range of this instrument as preset data and input it into the improved U2 -Net segmentation network (as shown in Figure 4) as a fixed hyperparameter. After obtaining the ratio of the pointer reading and the entire dial reading, calculate according to formula (1) to obtain the final instrument reading:
  • Step 2 Use the improved U 2 -Net segmentation network to obtain pointers and scales;
  • U 2 -Net proposes a unique RSU module (as shown in Figure 3) based on the encoding-decoding mode of the U-Net network structure.
  • This module can extract more different receptive field information from the context without increasing the overhead, which greatly improves the accuracy of segmentation, shortens the segmentation time, and realizes the function of real-time reading.
  • the grouped convolution GC GroupedConv
  • Figure 1 is used to replace the ordinary convolution of each layer, so as to achieve the purpose of reducing the number of parameters and completing reasoning faster;
  • Step 3 Transform the coordinates of the circular dial obtained by the U 2 -Net network, and use formula (2) to obtain the points on the corresponding rectangular position of r and ⁇ .
  • X cx+r*sin ⁇
  • Y cy-r*cos ⁇ (2)
  • X and Y are the coordinates after unfolding into a rectangle
  • cx and cy are the center coordinates of the ring
  • r is the radius
  • ⁇ is the rotation angle corresponding to the scale
  • Step 4 Use mean filtering to perform noise reduction on the scale array.
  • Mean filtering is a linear filter that calculates the average value of pixels in a window area, and then sets the calculated mean in the window as the pixel value at the current anchor point.
  • Mean filtering is implemented using convolution, and the obtained pixel value needs to be divided by the window size.
  • Mean filtering is simple to operate and highly efficient. It can effectively reduce noise for relatively simple segmented images such as pointers and scales, thereby better determining the position of the pointer relative to the scale, and finally reading according to the preset data.
  • step 1 the preset data, such as dial range, graduation value and other information, are input as hyperparameters into the improved U 2 -Net network, so as to obtain the scale corresponding to the pointer and then obtain the indication by calculating the proportion of the pointer passing through the scale.
  • the preset data such as dial range, graduation value and other information
  • step 2 the specific operation of segmentation is to first use labelme to mark the data and then convert the jpg format instrument image into a single-channel png format image, with 0 for background, 1 for pointer, and 2 for scale. Then, the original image and the corresponding png image are input into the network for training, and then the obtained scale and pointer are compared with the corresponding png image, and the gap between the two is minimized by continuously adjusting the parameters.
  • step 3 the center of the image is used as the origin and a certain width is scanned to expand the scale and pointer of the instrument panel into a rectangular image. Then, the two images are vertically superimposed to be converted into a one-dimensional array and drawn as a line graph.
  • step 4 the obtained image is subjected to noise reduction processing by mean filtering to facilitate clearer reading in the later stage; then, by locating each peak point, the number of scales on the dial and the scale can be obtained.
  • the position of the pointer can be located on the dial, and the percentage value of the dial can be obtained by calculation according to the preset data.
  • the method should be based on a camera, and the obtained readings can be read in real time at the back end.
  • a fixed camera needs to be installed at a densely populated area of instruments, and then the obtained picture is transmitted to the program.
  • the scale and pointer are first separated by an improved U2 -Net network and saved as png format images. Then, the annular scale image is converted into a rectangular image by coordinate transformation. Then, the image is subjected to noise reduction processing by mean filtering to improve recognition accuracy. Finally, the scales passed by the pointer and the ratio of all scales are obtained, and then multiplied by the range to obtain the final reading.
  • this embodiment provides an instrument identification device based on an improved U2 type network, including a processor and a storage medium;
  • the storage medium is used to store instructions
  • the processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.
  • this embodiment provides a storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.
  • the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
  • a computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.
  • These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biophysics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

本发明公开了一种基于改进U2型网络的仪表识别方法，包括：在RSU的基础上，用分组卷积Grouped Conv替换各层的普通卷积，用此网络分割出表盘和指针；通过传统数学方式将表盘展开为矩阵图像后将二维图像转换为一维数组；利用均值滤波对刻度值数组进行降噪处理；利用峰值定位出指针对应刻度值的位置，根据刻度值和预设数据输出示数；利用改进U2型网络对指针式仪表进行分割，再对得到的表盘通过传统计算机视觉方法实现自动读数；指针式仪表结构简单、安装方便且对电磁干扰等工业因素不敏感，故此方法相比人工读数拥有精度高、可靠性高、读数快且成本低等优点，可以极大提升工作效率。

Description

一种基于改进U2型网络的仪表识别方法 技术领域
本发明涉及一种基于改进U2型网络的仪表识别方法，属于计算机视觉的图像处理技术领域。
背景技术
指针式仪表作为工业上最长用的仪表，具有可靠，维护简单，价格低廉等诸多优点，同时可以适应各种不同场景，如高压、高温、高辐射等环境；但由于其工作环境的严苛性，读表操作通常无法通过人工完成，因此自动读数成为了解决该问题的方法。
发明内容
基于深度学习的图像分割技术对外界干扰因素有更强的稳定性，达到更精确的自动读数，从而能够保证工业生产过程中的实时监测；这种方案既不需要对传统车间进行大规模整改就能够达到节约时间、节省人力物力的目的，还能够达到比较良好的自动识别读数效果；
U2-Net是一个两层嵌套的U型网络结构，主要包括提取多尺度特征提取的残差U型块(residual U-blocks,RSU)和连接RSU的外层U型结构；该算法在设计过程中无需使用图像分类的预训练主干网络，在低计算成本的前提下可以提取更高分辨率的特征图；其中，RSU由三个部分构成：对输入进行通道转换的输入卷积层、多尺度提取上下文信息的U型结构以及通过残差块连接输入层和中间层的残差连接层；
Grouped Conv最早是在AlexNet中出现，由于当时硬件发展不够导致无法 在一个GPU通道下进行运算，所以将特征图分给两个GPU进行计算，之后再将两个计算结果进行拼接操作(concatenation)；简单来说就是将输入通道H*W*C1的深度C1平均分成g组，即每组个，同时C2个h*w*C1卷积滤波器也被分成了g组，即每组个的卷积核，每组中的每个滤波器只和前层输入的1/g个特征图进行卷积操作，之后再进行拼接操作，如图1所示；可以简单计算一下，对于没有进行Grouped Conv的卷积来说，输入为H*W*C1，卷积核大小为C2*h*w*C1，其参数量即为C2*h*w*C1，而对于Grouped Conv来说，输入特征图大小为H*W*C1/g，因此需要C2个H*W*C1/g大小的卷积核，由于拼接操作不需要额外的参数量，所以参数量为H*W*C1/g，为原来的1/g。
为实现以上目的，本发明采用以下技术方案：
第一方面，提供一种基于改进U2型网络的仪表识别方法，包括：
获取待识别的仪表图像和仪表预设数据；
将待识别的仪表图像输入训练好的基于改进U2型网络的图像分割模型，根据所述图像分割模型的输出，得到包含有表盘刻度和指针图像的带有掩码的图片；
将带有掩码的图片上环形表盘刻度展开为二维矩阵图像，再将二维矩阵图像转换为一维刻度数组；
对所述刻度数组使用均值滤波进行降噪处理得到指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums；
基于指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums，计算得到最终的仪表读数，作为仪表识别结果。
在一些实施例中，将带有掩码的图片上环形表盘刻度展开为二维矩阵图像， 包括：
X＝cx+r*sinθ
Y＝cy-r*cosθ
其中X、Y是展开为矩形之后的坐标，cx和cy为环形的中心坐标，r为半径，θ为刻度对应的旋转角度。
在一些实施例中，基于指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums，计算得到最终的仪表读数，包括：
其中Num表示仪表读数，Point_num表示指针所经过的表盘刻度个数，Nums表示总的表盘刻度，R是该仪表的量程。
在一些实施例中，所述基于改进U2型网络的图像分割模型包括提取多尺度特征提取的残差U型块RSU和连接RSU的外层U型结构；其中，RSU由三个部分构成：对输入进行通道转换的输入卷积层、多尺度提取上下文信息的U型结构以及通过残差块连接输入层和中间层的残差连接层；其中残差U型块RSU中的卷积采用分组卷积Grouped Conv；对于分组卷积Grouped Conv来说，输入特征图大小为H*W*C1/g，因此需要C2个H*W*C1/g大小的卷积核，由于拼接操作不需要额外的参数量，所以参数量为H*W*C1/g，为原来的1/g。
在一些实施例中，所述基于改进U2型网络的图像分割模型的训练方法包括：
获得带标签的仪表图像数据集，将数据集分为训练集、验证集和测试集，所述标签包括指针、表盘刻度和背景；
利用所述数据集对基于改进U2型网络的图像分割模型进行训练、测试，得到训练好的基于改进U2型网络的图像分割模型。
在一些实施例中，利用所述数据集对基于改进U2型网络的图像分割模型进行训练，包括：
利用labelme做数据标记后将jpg格式仪表图片转成单通道png格式图片，且0为背景、1为指针、2为刻度，之后将原图和对应的png图片输入到网络中进行训练，然后将得到的刻度和指针与对应的png图片进行比较，通过不断调整参数使两者之间的差距最小化，得到训练好的基于改进U2型网络的图像分割模型。
在一些实施例中，所述仪表预设数据包括同一仪表的刻度值、分度值和量程信息。
第二方面，本发明提供了一种基于改进U2型网络的仪表识别装置，包括处理器及存储介质；
所述存储介质用于存储指令；
所述处理器用于根据所述指令进行操作以执行根据第一方面所述方法的步骤。
第三方面，本发明提供了一种存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现第一方面所述方法的步骤。
有益效果：本发明提出的一种基于改进U2型网络的仪表识别方法，为减弱下采样带来的细节特征丢失，提升对指针分割的效果，同时减少网络的参数量，在RSU的基础上，用GC(Grouped Conv)替换各层的普通卷积，用此网络分割出表盘和指针；通过传统数学方式将表盘展开为矩阵图像后将二维图像转换为一维数组；利用均值滤波对刻度值数组进行降噪处理；利用峰值定位出指针对应刻度值的位置，根据刻度值和预设数据输出示数；本发明主要利用改进U2型网 络对指针式仪表进行分割，再对得到的表盘通过传统计算机视觉方法实现自动读数；指针式仪表结构简单、安装方便且对电磁干扰等工业因素不敏感，故此方法相比人工读数拥有精度高、可靠性高、读数快且成本低等优点，可以极大提升工作效率。通过图像分割技术得到指针式仪表图像中的刻度和指针，之后根据预设的表盘信息进行计算得到最后示数，实现无人实时在线读数，提高了工作效率、节省了人力物力。
附图说明
图1是本发明实施例的Grouped Conv操作示意图；
图2是本发明实施例提供的方法流程图；
图3是U2-Net分割网络提出的RSU模块；
图4是U2-Net分割网络结构图；
图5是改进后的RSU模块。
具体实施方式
下面结合附图和具体实施例对本发明作进一步详细描述：
以下实施例仅用于更加清楚地说明本发明的技术方案，而不能以此来限制本发明的保护范围。
在本发明的描述中，若干的含义是一个以上，多个的含义是两个以上，大于、小于、超过等理解为不包括本数，以上、以下、以内等理解为包括本数。如果有描述到第一、第二只是用于区分技术特征为目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量或者隐含指明所指示的技术特征的先后关系。
本发明的描述中，参考术语“一个实施例”、“一些实施例”、“示意性实施例”、 “示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
实施例1
一种基于改进U2型网络的仪表识别方法，包括：
获取待识别的仪表图像和仪表预设数据；
将待识别的仪表图像输入训练好的基于改进U2型网络的图像分割模型，根据所述图像分割模型的输出，得到包含有表盘刻度和指针图像的带有掩码的图片；
将带有掩码的图片上环形表盘刻度展开为二维矩阵图像，再将二维矩阵图像转换为一维刻度数组；
对所述刻度数组使用均值滤波进行降噪处理得到指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums；
基于指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums，计算得到最终的仪表读数，作为仪表识别结果。
在一些实施例中，将带有掩码的图片上环形表盘刻度展开为二维矩阵图像，包括：
X＝cx+r*sinθ
Y＝cy-r*cosθ
其中X、Y是展开为矩形之后的坐标，cx和cy为环形的中心坐标，r为半径， θ为刻度对应的旋转角度。
在一些实施例中，基于指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums，计算得到最终的仪表读数，包括：
其中Num表示仪表读数，Point_num表示指针所经过的表盘刻度个数，Nums表示总的表盘刻度，R是该仪表的量程。
在一些实施例中，所述基于改进U2型网络的图像分割模型包括提取多尺度特征提取的残差U型块RSU和连接RSU的外层U型结构；其中，RSU由三个部分构成：对输入进行通道转换的输入卷积层、多尺度提取上下文信息的U型结构以及通过残差块连接输入层和中间层的残差连接层；其中残差U型块RSU中的卷积采用分组卷积Grouped Conv；对于分组卷积Grouped Conv来说，输入特征图大小为H*W*C1/g，因此需要C2个H*W*C1/g大小的卷积核，由于拼接操作不需要额外的参数量，所以参数量为H*W*C1/g，为原来的1/g。
在一些实施例中，所述基于改进U2型网络的图像分割模型的训练方法包括：
获得带标签的仪表图像数据集，将数据集分为训练集、验证集和测试集，所述标签包括指针、表盘刻度和背景；
利用所述数据集对基于改进U2型网络的图像分割模型进行训练、测试，得到训练好的基于改进U2型网络的图像分割模型。
在一些实施例中，利用所述数据集对基于改进U2型网络的图像分割模型进行训练，包括：
利用labelme做数据标记后将jpg格式仪表图片转成单通道png格式图片，且0为背景、1为指针、2为刻度，之后将原图和对应的png图片输入到网络中 进行训练，然后将得到的刻度和指针与对应的png图片进行比较，通过不断调整参数使两者之间的差距最小化，得到训练好的基于改进U2型网络的图像分割模型。
在一些实施例中，所述仪表预设数据包括同一仪表的刻度值、分度值和量程信息。
如图2所示，一种基于改进U2型网络的仪表识别方法，包含以下步骤：
步骤1：获得实拍得到的同一指针式仪表图像数据集，将此数据集按6：2：2的比例分为训练集、验证集和测试集，同时记录这种仪表的刻度值、分度值和量程等固定信息作为预设数据输入到改进后的U2-Net分割网络(如图4所示)中作为固定的超参数，为了在得到指针示数和整个表盘示数的比例之后按照公式(1)进行计算得到最终仪表读数：
其中Num表示仪表读数，Point_num表示指针所经过的表盘刻度个数，Nums表示总的表盘刻度，R是该仪表的量程；
步骤2：使用改进后的U2-Net分割网络得到指针和刻度；U2-Net在U-Net网络结构的编码-解码模式基础上，提出了一种独特的RSU模块(如图3所示)，该模块能够在不增加开销的情况下从上下文中提取更多不同的感受野信息，这极大提高了分割的准确度，也缩短了分割时间，实现了实时读数的功能，在RSU的基础上，用如图1所示的分组卷积GC(GroupedConv)替换各层的普通卷积，从而达到减少参数量的目的，更快完成推理；
步骤3：将上述通过U2-Net网络得到的环形表盘刻度通过坐标变换，将r和θ通过公式(2)得到对应矩形位置上的点
X＝cx+r*sinθ
Y＝cy-r*cosθ               (2)
其中X、Y是展开为矩形之后的坐标，cx和cy为环形的中心坐标，r为半径，θ为刻度对应的旋转角度；
步骤4：对上述刻度数组使用均值滤波进行降噪处理，均值滤波是一种线性滤波器，将一个窗口区域中的像素计算平均值，然后将窗口中计算得到的均值设置为当前锚点上的像素值，均值滤波用卷积实现，得到的像素值需除以窗口尺寸大小；均值滤波操作简单，效率高，对于指针和刻度这类较为单一分割图像能够很好地降噪，从而更好确定指针相对刻度的位置，最后根据预设数据进行读数。
所述步骤1中，将所述预设数据，如表盘量程、分度值等信息作为超参数输入到改进的U2-Net网络中，便于得到指针对应的刻度之后通过计算指针经过刻度的比例来获得示数。
所述步骤2中，分割的具体操作是先利用labelme做数据标记后将jpg格式仪表图片转成单通道png格式图片，且0为背景、1为指针、2为刻度，之后将原图和对应的png图片输入到网络中进行训练，然后将得到的刻度和指针与对应的png图片进行比较，通过不断调整参数使两者之间的差距最小化。
所述步骤3中，以图像中心为原点、一定宽度进行扫描，将仪表盘的刻度和指针展开为矩形的图像，之后将两张图像通过竖向叠加，转变为一维数组，绘制为折线图。
所述步骤4中，对得到的图像通过均值滤波进行降噪处理，便于后期更清楚读数；之后通过定位每一个尖峰的点，即可得刻度盘的刻度数量以及刻度的 位置，并可定位到指针在刻度盘中的相对位置，再根据预设数据计算即可得到该表盘的百分比数值。
所述方法应基于摄像头，同时在后端能够实时对所得到的示数；以化工厂为例，需要在仪表密集处安装固定摄像头，然后将得到的画面传输到程序中，先通过改进后的U2-Net网络将刻度和指针分割出来，保存为png格式图片；之后通过坐标变换将环形刻度图像转换为矩形图像；之后对图片通过均值滤波进行降噪处理，提高识别精度，最后得到指针所经过的刻度以及所有刻度的比值，然后乘上量程即为最终读数。
实施例2
第二方面，本实施例提供了一种基于改进U2型网络的仪表识别装置，包括处理器及存储介质；
所述存储介质用于存储指令；
所述处理器用于根据所述指令进行操作以执行根据实施例1所述方法的步骤。
实施例3
第三方面，本实施例提供了一种存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现实施例1所述方法的步骤。
本领域内的技术人员应明白，本申请的实施例可提供为方法、系统、或计算机程序产品。因此，本申请可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本申请可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本申请是参照根据本申请实施例的方法、设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理设备上，使得在计算机或其他可编程设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的步骤。
以上所述仅是本发明的优选实施方式，应当指出：对于本技术领域的普通技术人员来说，在不脱离本发明原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本发明的保护范围。

Claims (9)

  1. 一种基于改进U2型网络的仪表识别方法，其特征在于，包括以下步骤：
    获取待识别的仪表图像和仪表预设数据；
    将待识别的仪表图像输入训练好的基于改进U2型网络的图像分割模型，根据所述图像分割模型的输出，得到包含有表盘刻度和指针图像的带有掩码的图片；
    将带有掩码的图片上环形表盘刻度展开为二维矩阵图像，再将二维矩阵图像转换为一维刻度数组；
    对所述刻度数组使用均值滤波进行降噪处理得到指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums；
    基于指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums，计算得到最终的仪表读数，作为仪表识别结果。
  2. 根据权利要求1所述的基于改进U2型网络的仪表识别方法，其特征在于，将带有掩码的图片上环形表盘刻度展开为二维矩阵图像，包括：
    X＝cx+r*sinθ
    Y＝cy-r*cosθ
    其中X、Y是展开为矩形之后的坐标，cx和cy为环形的中心坐标，r为半径，θ为刻度对应的旋转角度。
  3. 根据权利要求1所述的基于改进U2型网络的仪表识别方法，其特征在于，基于指针所经过的表盘刻度个数Point_num和总的表盘刻度Nums，计算得到最终的仪表读数，包括：
    其中Num表示仪表读数，Point_num表示指针所经过的表盘刻度个数，Nums表示总的表盘刻度，R是该仪表的量程。
  4. 根据权利要求1所述的基于改进U2型网络的仪表识别方法，其特征在于，
    所述基于改进U2型网络的图像分割模型包括提取多尺度特征提取的残差U型块RSU和连接RSU的外层U型结构；其中，RSU由三个部分构成：对输入进行通道转换的输入卷积层、多尺度提取上下文信息的U型结构以及通过残差块连接输入层和中间层的残差连接层；其中残差U型块RSU中的卷积采用分组卷积Grouped Conv；对于分组卷积Grouped Conv来说，输入特征图大小为H*W*C1/g，因此需要C2个H*W*C1/g大小的卷积核，由于拼接操作不需要额外的参数量，所以参数量为H*W*C1/g，为原来的1/g。
  5. 根据权利要求1所述的基于改进U2型网络的仪表识别方法，其特征在于，所述基于改进U2型网络的图像分割模型的训练方法包括：
    获得带标签的仪表图像数据集，将数据集分为训练集、验证集和测试集，所述标签包括指针、表盘刻度和背景；
    利用所述数据集对基于改进U2型网络的图像分割模型进行训练、测试，得到训练好的基于改进U2型网络的图像分割模型。
  6. 根据权利要求5所述的基于改进U2型网络的仪表识别方法，其特征在于，利用所述数据集对基于改进U2型网络的图像分割模型进行训练，包括：
    利用labelme做数据标记后将jpg格式仪表图片转成单通道png格式图片，且0为背景、1为指针、2为刻度，之后将原图和对应的png图片输入到网络中进行训练，然后将得到的刻度和指针与对应的png图片进行比较，通过不断调整参数使两者之间的差距最小化，得到训练好的基于改进U2型网络的图像分割 模型。
  7. 根据权利要求1所述的基于改进U2型网络的仪表识别方法，其特征在于，所述仪表预设数据包括同一仪表的刻度值、分度值和量程信息。
  8. 一种基于改进U2型网络的仪表识别装置，其特征在于，包括处理器及存储介质；
    所述存储介质用于存储指令；
    所述处理器用于根据所述指令进行操作以执行根据权利要求1至7任一项所述方法的步骤。
  9. 一种存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现权利要求1至7任一项所述方法的步骤。
PCT/CN2023/103126 2022-11-01 2023-06-28 一种基于改进u2型网络的仪表识别方法 WO2024093287A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/469,583 US20240005639A1 (en) 2022-11-01 2023-09-19 Instrument recognition method based on improved u2 network

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211354710.5 2022-11-01
CN202211354710.5A CN115690500A (zh) 2022-11-01 2022-11-01 一种基于改进u2型网络的仪表识别方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/469,583 Continuation US20240005639A1 (en) 2022-11-01 2023-09-19 Instrument recognition method based on improved u2 network

Publications (1)

Publication Number Publication Date
WO2024093287A1 true WO2024093287A1 (zh) 2024-05-10

Family

ID=85048018

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/103126 WO2024093287A1 (zh) 2022-11-01 2023-06-28 一种基于改进u2型网络的仪表识别方法

Country Status (2)

Country Link
CN (1) CN115690500A (zh)
WO (1) WO2024093287A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118429953A (zh) * 2024-07-04 2024-08-02 成都考拉悠然科技有限公司 仪表盘识别读数方法及装置

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115690500A (zh) * 2022-11-01 2023-02-03 南京邮电大学 一种基于改进u2型网络的仪表识别方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113239930A (zh) * 2021-05-14 2021-08-10 广州广电运通金融电子股份有限公司 一种玻璃纸缺陷识别方法、系统、装置及存储介质
CN113496494A (zh) * 2021-06-17 2021-10-12 北京理工大学 基于drr模拟数据生成的二维骨骼分割方法及装置
US20220012579A1 (en) * 2021-09-24 2022-01-13 Masayoshi Asama Neural network accelerator system for improving semantic image segmentation
CN114549981A (zh) * 2022-02-11 2022-05-27 国网河南省电力公司电力科学研究院 一种基于深度学习的智能巡检指针式仪表识别及读数方法
CN115131579A (zh) * 2022-07-24 2022-09-30 河南垂天科技有限公司 轻量化的残差神经网络模型
CN115239952A (zh) * 2022-06-14 2022-10-25 天津理工大学 基于YOLOV5及U2-Net框架辅助读取工业表的方法
CN115690500A (zh) * 2022-11-01 2023-02-03 南京邮电大学 一种基于改进u2型网络的仪表识别方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113239930A (zh) * 2021-05-14 2021-08-10 广州广电运通金融电子股份有限公司 一种玻璃纸缺陷识别方法、系统、装置及存储介质
CN113496494A (zh) * 2021-06-17 2021-10-12 北京理工大学 基于drr模拟数据生成的二维骨骼分割方法及装置
US20220012579A1 (en) * 2021-09-24 2022-01-13 Masayoshi Asama Neural network accelerator system for improving semantic image segmentation
CN114549981A (zh) * 2022-02-11 2022-05-27 国网河南省电力公司电力科学研究院 一种基于深度学习的智能巡检指针式仪表识别及读数方法
CN115239952A (zh) * 2022-06-14 2022-10-25 天津理工大学 基于YOLOV5及U2-Net框架辅助读取工业表的方法
CN115131579A (zh) * 2022-07-24 2022-09-30 河南垂天科技有限公司 轻量化的残差神经网络模型
CN115690500A (zh) * 2022-11-01 2023-02-03 南京邮电大学 一种基于改进u2型网络的仪表识别方法

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118429953A (zh) * 2024-07-04 2024-08-02 成都考拉悠然科技有限公司 仪表盘识别读数方法及装置

Also Published As

Publication number Publication date
CN115690500A (zh) 2023-02-03

Similar Documents

Publication Publication Date Title
WO2024093287A1 (zh) 一种基于改进u2型网络的仪表识别方法
CN109635806B (zh) 基于残差网络的电表数值识别方法
WO2021109775A1 (zh) 训练样本生成、模型训练、字符识别方法及其装置
CN114549981A (zh) 一种基于深度学习的智能巡检指针式仪表识别及读数方法
CN111598869B (zh) 一种对显示屏幕的Mura进行检测的方法、设备及存储介质
CN108053416B (zh) 一种基于单幅卫星图像的最大储油量提取系统
CN112819748B (zh) 一种带钢表面缺陷识别模型的训练方法及装置
CN110288612B (zh) 铭牌定位与校正方法及设备
CN109508709B (zh) 一种基于机器视觉的单指针仪表读数方法
CN110245597B (zh) 一种指针式仪表通用性识别方法
CN112037182B (zh) 基于时序图像的机车走行部故障检测方法、装置及存储介质
CN115082658A (zh) 一种指针表读数的识别方法及装置
CN116543376B (zh) 一种指针式仪表读数方法和系统
CN116310285B (zh) 一种基于深度学习的指针式仪表自动读数方法及系统
CN115239952A (zh) 基于YOLOV5及U2-Net框架辅助读取工业表的方法
CN112990190A (zh) 指针式仪表读数的识别方法、装置
CN116678893A (zh) 一种基于深度学习的热轧带钢表面缺陷检测方法
US20240005639A1 (en) Instrument recognition method based on improved u2 network
CN114693705A (zh) 一种指针式仪表测量值读取方法、装置及系统
CN114638961A (zh) 一种指针表盘识别方法、系统及计算机存储介质
CN110222688B (zh) 一种基于多层次相关滤波的仪表定位方法
CN112183661A (zh) 一种基于细节注意力机制的仪表自动化识别方法
CN118334351B (zh) 一种基于深度学习的指针仪表读数识别方法
CN104156696A (zh) 基于双方向图的快速局部不变特征描述子的构造方法
CN117237777B (zh) 一种基于多模态融合的船舶目标识别方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23884226

Country of ref document: EP

Kind code of ref document: A1