CN109840900B - A fault online detection system and detection method applied to intelligent manufacturing workshops - Google Patents

A fault online detection system and detection method applied to intelligent manufacturing workshops Download PDF

Info

Publication number
CN109840900B
CN109840900B CN201811651018.2A CN201811651018A CN109840900B CN 109840900 B CN109840900 B CN 109840900B CN 201811651018 A CN201811651018 A CN 201811651018A CN 109840900 B CN109840900 B CN 109840900B
Authority
CN
China
Prior art keywords
image
manufacturing
neural network
area
workpiece
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201811651018.2A
Other languages
Chinese (zh)
Other versions
CN109840900A (en
Inventor
沈治
朱丽霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Changzhou Vocational Institute of Light Industry
Original Assignee
Changzhou Vocational Institute of Light Industry
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Changzhou Vocational Institute of Light Industry filed Critical Changzhou Vocational Institute of Light Industry
Priority to CN201811651018.2A priority Critical patent/CN109840900B/en
Publication of CN109840900A publication Critical patent/CN109840900A/en
Application granted granted Critical
Publication of CN109840900B publication Critical patent/CN109840900B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to a fault online detection system and a fault online detection method applied to an intelligent manufacturing workshop, which construct a manufacturing defect prediction model based on a deep neural network, effectively utilize a large number of typical manufacturing defects obtained in the manufacturing process of the actual intelligent manufacturing workshop, and perform training and learning on a standard sample image library formed by typical manufacturing defect images by combining image acquisition and image processing technology, so that the deep neural network model can be used for identifying and classifying the manufacturing defects in real time, dynamically obtain manufacturing information of the product through real-time image acquisition, processing and analysis in the manufacturing process of the product, and provide and automatically execute maintenance strategies of the manufacturing defects by combining a PLC (programmable logic controller) and a historical maintenance database, and can provide more accurate reference information for product manufacturing precision and manufacturing information collection and analysis of the intelligent manufacturing workshop.

Description

一种应用于智能制造车间的故障在线检测系统及检测方法A fault online detection system and detection method applied to intelligent manufacturing workshops

技术领域Technical field

本发明属于智能制造技术领域,具体地,涉及一种应用于智能制造车间的故障在线检测系统及其方法。The invention belongs to the field of intelligent manufacturing technology, and specifically relates to an online fault detection system and method applied in an intelligent manufacturing workshop.

背景技术Background technique

目前的智能制造车间,由于采用的高速、高精度的数控机床较多,加工制造的产品允许误差小,以及智能制造车间对人员综合素质、环境都具有较高的要求,使得制造制造车间的产品制造过程中的制造缺陷有时难以被及时人工发现,通常只能通过后序工序的人工检验才得以发现,从而造成智能制造的废品增加、生产率下降等问题,严重时,甚至会造成后序制造机床的报废,这在智能制造装配区间已经有所发生,因此,对制造制造车间的产品在制造过程中进行动态的制造缺陷检验已经诊断,是影响智能制造进一步发展的阻碍因素之一,如今也得到了业内学者的广泛研究。The current intelligent manufacturing workshops use many high-speed and high-precision CNC machine tools, the allowable errors of processed and manufactured products are small, and the intelligent manufacturing workshops have high requirements for the comprehensive quality of personnel and the environment, making the products in the manufacturing workshops Manufacturing defects in the manufacturing process are sometimes difficult to detect manually in time and can usually only be discovered through manual inspection in subsequent processes. This causes problems such as an increase in waste products and a decrease in productivity in intelligent manufacturing. In severe cases, it may even cause subsequent manufacturing of machine tools. Scrapped, which has already occurred in the intelligent manufacturing assembly area. Therefore, dynamic manufacturing defect inspection and diagnosis of products in the manufacturing workshop during the manufacturing process is one of the hindering factors affecting the further development of intelligent manufacturing. It has also been Extensive research by scholars in the industry.

发明内容Contents of the invention

本发明的目的是提供一种应用于智能制造车间的故障在线检测系统及检测方法。The purpose of the present invention is to provide an online fault detection system and detection method applied in intelligent manufacturing workshops.

根据本发明的上述目的,提出一种故障在线检测系统,包括工件检测平台、图像采集单元、图像处理单元、特征向量提取单元、深度神经网络单元以及计算机控制单元,其中,According to the above object of the present invention, an online fault detection system is proposed, including a workpiece detection platform, an image acquisition unit, an image processing unit, a feature vector extraction unit, a deep neural network unit and a computer control unit, wherein,

所述工件检测平台包括检测工位,所述图像采集单元内的面阵相机采集检测工位上工件检测平台的图像,并发送至图像处理单元;The workpiece detection platform includes a detection station, and the area array camera in the image acquisition unit collects images of the workpiece detection platform on the detection station and sends them to the image processing unit;

所述图像处理单元对接收到的图像进行分辨率扫描,获得当前检测工位的敏感区域图像,且对敏感区域图像进行去噪,再将去噪后的敏感区域图像发送至特征向量提取单元;The image processing unit performs a resolution scan on the received image to obtain the sensitive area image of the current detection station, denoises the sensitive area image, and then sends the denoised sensitive area image to the feature vector extraction unit;

特征向量提取单元对敏感区域图像进行边缘检测,形成目标区域,并分别通过公式(1)至(3)计算获得目标区域的边缘面积、边缘形状因子以及目标区域平均半径,再加上前3维的Hu不变矩,构成具有四个特征变量的敏感区域的特征向量,以反映当前工件检测平台的工件质量信息,特征向量作为输入层发送至深度神经网络单元;The feature vector extraction unit performs edge detection on the sensitive area image to form a target area, and calculates the edge area, edge shape factor and average radius of the target area through formulas (1) to (3) respectively, plus the first 3 dimensions The Hu invariant moments form a feature vector of the sensitive area with four feature variables to reflect the workpiece quality information of the current workpiece detection platform. The feature vector is sent to the deep neural network unit as an input layer;

上式中,参数M和N为目标区域的边缘点个数,其中t(x,y)为各边缘点的灰度值;参数L为目标区域的周长,可采用图像处理技术中的链码法进行计算获得,参考K为目标区域边界上的边缘点个数,(xk,yk)表示位于目标区域边界上的像素坐标,/>表示目标区域的质心坐标,可通过如下公式进行计算:In the above formula, the parameters M and N are the number of edge points in the target area, Among them, t(x,y) is the gray value of each edge point; the parameter L is the perimeter of the target area, which can be calculated using the chain code method in image processing technology. The reference K is the number of edge points on the boundary of the target area. Number, (x k ,y k ) represents the pixel coordinates located on the boundary of the target area,/> Represents the centroid coordinates of the target area, which can be calculated by the following formula:

其中,参数A表示敏感区域的面积,且适于在图像处理中识别到敏感区域时获得其大小;Among them, parameter A represents the area of the sensitive area, and is suitable for obtaining its size when a sensitive area is identified in image processing;

深度神经网络单元基于神经网络算法构建制造缺陷预测模型,对工件检测平台的图像特征向量进行训练、学习与分类,识别当前工件检测平台上的待测工件的制造缺陷类型,并将分类结果反馈给计算机控制单元。The deep neural network unit builds a manufacturing defect prediction model based on the neural network algorithm, trains, learns and classifies the image feature vectors of the workpiece detection platform, identifies the manufacturing defect type of the workpiece to be tested on the current workpiece detection platform, and feeds the classification results back to Computer control unit.

又一方面,本发明还提供了一种应用于智能制造车间的故障在线检测方法,包括:On the other hand, the present invention also provides an online fault detection method applied in intelligent manufacturing workshops, including:

步骤1:构建基于深度神经网络的制造缺陷预测模型,通过样本图像对深度神经网络进行训练学习;Step 1: Construct a manufacturing defect prediction model based on a deep neural network, and train the deep neural network through sample images;

步骤2:工件检测平台在计算机控制模块的控制指令下带动待测工件在检测工位上运动使其到达预设的检测位置,触发接近开关,并发送触发信号至计算机控制单元;Step 2: Under the control instructions of the computer control module, the workpiece detection platform drives the workpiece to be tested to move on the detection station to reach the preset detection position, triggers the proximity switch, and sends the trigger signal to the computer control unit;

步骤3:计算机控制单元分别向LED面光源矩阵和图像采集单元发送指令,所述LED面光源矩阵打开照明,所述图像采集单元对待测工件进行拍摄,并将生成的图像发送至图像处理单元;Step 3: The computer control unit sends instructions to the LED surface light source matrix and the image acquisition unit respectively. The LED surface light source matrix turns on the illumination. The image acquisition unit photographs the workpiece to be tested and sends the generated image to the image processing unit;

步骤4:图像处理单元识别并分割出当前检测工位的敏感区域,针对该局部区域进行图像去噪处理,去噪后将该敏感区域图像发送至特征向量提取单元;Step 4: The image processing unit identifies and segments the sensitive area of the current detection station, performs image denoising on the local area, and sends the image of the sensitive area to the feature vector extraction unit after denoising;

步骤5:特征向量提取单元对敏感区域进行边缘检测,形成目标区域,并计算获得目标区域的边缘面积、边缘形状因子以及目标区域平均半径,结合前3维的Hu不变矩,构成具有四个特征变量的敏感区域的特征向量;Step 5: The feature vector extraction unit performs edge detection on the sensitive area to form a target area, and calculates the edge area, edge shape factor and average radius of the target area. Combined with the Hu invariant moments of the first three dimensions, it forms a four-dimensional Feature vector of the sensitive area of the feature variable;

步骤6:基于训练好的深度神经网络对特征向量进行制造信息的诊断,预测并分类待测工件的制造缺陷,并将分类结果反馈给计算机控制单元。Step 6: Diagnose the manufacturing information based on the trained deep neural network on the feature vector, predict and classify the manufacturing defects of the workpiece to be tested, and feed the classification results back to the computer control unit.

本发明的有益效果是:The beneficial effects of the present invention are:

(1)本发明首次将图像采集与图像处理技术应用于智能制造车间的产品制造缺陷信息识别与诊断中,通过基于深度神经网络构建的制造缺陷预测模型,对智能制造车间的产品进行实时图像采集、处理与特征提取,获得产品制造信息的特征向量表示,从而实现了在智能制造过程中的产品制造信息可显示、缺陷可诊断、维修策略可自动生成,形成产品制造的实时监测、缺陷结论的实时分析、维修策略的自动实施的有机一体化系统,从而有效地解决了现有的制约智能制造技术广泛应用的因素之一;(1) For the first time, this invention applies image acquisition and image processing technology to the identification and diagnosis of product manufacturing defect information in an intelligent manufacturing workshop. Through a manufacturing defect prediction model built based on a deep neural network, real-time image acquisition of products in the intelligent manufacturing workshop is performed. , processing and feature extraction to obtain the feature vector representation of product manufacturing information, thereby realizing that product manufacturing information can be displayed, defects can be diagnosed, and maintenance strategies can be automatically generated in the intelligent manufacturing process, forming real-time monitoring of product manufacturing and defect conclusions. An organic integrated system with real-time analysis and automatic implementation of maintenance strategies, thus effectively solving one of the existing factors that restrict the widespread application of intelligent manufacturing technology;

(2)本发明为了获得精准的图像处理结果,通过应用若干个光纤滑环以及图像处理过程中应用分辨率扫描技术,有效解决了智能制造车间中可能出现的机床与检测工位的微小振动带来的图像采集精度以及面阵相机的镜头偏转问题,提高了本发明的产品制造在线故障缺陷诊断与识别的精度;(2) In order to obtain accurate image processing results, this invention effectively solves the tiny vibration bands that may occur in machine tools and inspection stations in intelligent manufacturing workshops by applying several optical fiber slip rings and resolution scanning technology in the image processing process. The improved image acquisition accuracy and the lens deflection problem of the area array camera improve the accuracy of online fault defect diagnosis and identification of the product manufacturing of the present invention;

(3)本发明为了减少图像处理的耗时,基于待测工件在当前检测工位上可能出现的典型制造缺陷,针对性地设置背景标板的标记点位置,使得后序图像处理中可以直接定位出当前检测工位的敏感区域图像,针对这一局部区域进行图像处理与特征变量的提取与计算,极大地提高了图像处理效率,使得本发明的产品在线故障预测系统与方法相较于传统的对整个图像进行处理与特征变量的提取,显得更为实时与高效,极大的提高了本发明在实际智能制造车间的应用价值。(3) In order to reduce the time-consuming image processing, the present invention specifically sets the marking point position of the background mark based on the typical manufacturing defects that may occur in the workpiece to be tested at the current inspection station, so that the subsequent image processing can be directly The sensitive area image of the current detection station is located, and image processing and feature variable extraction and calculation are performed on this local area, which greatly improves the image processing efficiency and makes the product online fault prediction system and method of the present invention more efficient than traditional ones. The entire image processing and feature variable extraction are more real-time and efficient, which greatly improves the application value of the present invention in actual intelligent manufacturing workshops.

附图说明Description of drawings

下面结合附图和实施例对本发明进一步说明。The present invention will be further described below in conjunction with the accompanying drawings and examples.

图1是本发明的故障在线检测系统的示意图。Figure 1 is a schematic diagram of the fault online detection system of the present invention.

具体实施方式Detailed ways

现在结合附图对本发明作进一步详细的说明。这些附图均为简化的示意图,仅以示意方式说明本发明的基本结构,因此其仅显示与本发明有关的构成。The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams that only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the structures related to the present invention.

本发明提供了一种应用于智能制造车间的故障在线检测系统及其方法,其基于深度神经网络构建制造缺陷预测模型,有效利用大量实际智能制造车间制造过程中获得的典型制造缺陷,结合图像采集与图像处理技术,对典型制造缺陷图像构成的标准样本图像库进行训练学习,使得该深度神经网络模型可以用于实时的制造缺陷识别与分类,能够在产品的制造过程中,通过实时图像采集、处理与分析动态地获得产品的制造信息,并结合PLC控制器以及历史维修数据库,给出并自动执行制造缺陷的维修策略,能够为智能制造车间的产品制造精度、制造信息收集与分析提供更为准确的参考信息。The present invention provides an online fault detection system and method applied to intelligent manufacturing workshops. It builds a manufacturing defect prediction model based on a deep neural network, effectively utilizes a large number of typical manufacturing defects obtained in the manufacturing process of actual intelligent manufacturing workshops, and combines image acquisition with With image processing technology, a standard sample image library composed of typical manufacturing defect images is trained and learned, so that the deep neural network model can be used for real-time manufacturing defect identification and classification, and can be used in the product manufacturing process through real-time image collection, Processing and analysis dynamically obtain the manufacturing information of the product, and combine it with the PLC controller and historical maintenance database to provide and automatically execute the maintenance strategy for manufacturing defects, which can provide better product manufacturing accuracy, manufacturing information collection and analysis in the intelligent manufacturing workshop. Accurate reference information.

通过如下实施例对本发明的实现过程进行详细论述。The implementation process of the present invention is discussed in detail through the following examples.

如图1所示,本发明的参照附图1,本发明提供了一种应用于智能制造车间的故障在线检测系统,包括产品检测平台1、图像采集单元2、图像处理单元3、特征向量提取单元4、深度神经网络单元5以及计算机控制单元6。As shown in Figure 1, with reference to Figure 1, the present invention provides an online fault detection system applied to an intelligent manufacturing workshop, including a product detection platform 1, an image acquisition unit 2, an image processing unit 3, and feature vector extraction. Unit 4, deep neural network unit 5 and computer control unit 6.

在优选的实施例中,产品检测平台1的数量可以是若干个,每个产品检测平台1对应一个图像采集单元2,图像处理单元3、特征向量提取单元4以及深度神经网络单元5可以作为软件系统集成在计算机控制单元6中。In the preferred embodiment, the number of product detection platforms 1 can be several, each product detection platform 1 corresponds to an image acquisition unit 2, the image processing unit 3, the feature vector extraction unit 4 and the deep neural network unit 5 can be used as software The system is integrated in the computer control unit 6.

其中,产品检测平台1包括检测工位11、接近开关12以及智能定位与放行机构13,在优选的实施例中,接近开关12、智能定位与放行机构13均与计算机控制单元6通信连接,其中智能定位与放行机构13接收计算机控制单元6的控制指令,带动待测工件在检测工位11上运动使其到达预设的检测位置,触发接近开关12并发送触发信号至计算机控制单元6,由计算机控制单元6启动图像采集单元2进行图像采集。Among them, the product detection platform 1 includes a detection station 11, a proximity switch 12 and an intelligent positioning and release mechanism 13. In the preferred embodiment, the proximity switch 12 and the intelligent positioning and release mechanism 13 are all communicatively connected with the computer control unit 6, where The intelligent positioning and release mechanism 13 receives the control instructions from the computer control unit 6, drives the workpiece to be tested to move on the detection station 11 to reach the preset detection position, triggers the proximity switch 12 and sends a trigger signal to the computer control unit 6. The computer control unit 6 starts the image acquisition unit 2 to collect images.

另一方面,在检测工位11的一侧设置有沿着柔性生产线连续布置的背景标板14,背景标板14上设置有多个对应当前检测工位11的标记点,在优选的实施例中,基于当前检测工位的待测工件可能出现的典型制造缺陷信息,设置背景标板14上的标记点位置,以便于后续图像处理对敏感区域的识别与分割。此外,检测工位11的另一侧设置有LED面光源矩阵15,该LED面光源矩阵15与计算机控制单元6通信连接,计算机控制单元6可以实现智能控制整个生产线上的LED面光源矩阵15的点亮与熄灭,在待测工件运动到当前检测工位11的预设位置时,计算机控制单元6发送指令给对应该检测工位11的LED面光源矩阵15,使其点亮,当待测工件被输送至下一个检测工位时,控制当前检测工位11的LED面光源矩阵15熄灭,背景标板14与LED面光源矩阵15共同构成背光照明环境,有利于待测工件在图像上与背景标板14的灰度对比,减少图像处理时的边缘检测耗时。On the other hand, a background mark 14 arranged continuously along the flexible production line is provided on one side of the detection station 11. A plurality of marking points corresponding to the current detection station 11 are provided on the background mark 14. In the preferred embodiment , based on the typical manufacturing defect information that may occur in the workpiece to be tested at the current inspection station, the position of the mark point on the background target plate 14 is set to facilitate the identification and segmentation of sensitive areas in subsequent image processing. In addition, an LED surface light source matrix 15 is provided on the other side of the detection station 11. The LED surface light source matrix 15 is communicatively connected to the computer control unit 6. The computer control unit 6 can realize intelligent control of the LED surface light source matrix 15 on the entire production line. On and off, when the workpiece to be tested moves to the preset position of the current detection station 11, the computer control unit 6 sends an instruction to the LED surface light source matrix 15 corresponding to the detection station 11 to make it light up. When the workpiece is transported to the next inspection station, the LED surface light source matrix 15 of the current inspection station 11 is controlled to be extinguished. The background target plate 14 and the LED surface light source matrix 15 together form a backlight lighting environment, which is conducive to the image of the workpiece to be tested and the The grayscale contrast of the background target plate 14 reduces the time-consuming edge detection during image processing.

图像采集单元2包括依次通信连接的面阵相机21、第一光电转换元件22、光纤滑环23以及第二光电转换元件24,其中,当待测工件在检测工位11上运动到预设的检测位置并接近面阵相机21的拍摄视野中心时,触发接近开关12并发送触发信号至计算机控制单元6;面阵相机21的感光镜头轴线与检测工位流转方向垂直设置,考虑到智能制造车间中,设备的运转可能会被检测工位产生一定的振动,从而影响与检测工位固定连接的面阵相机21的拍摄精度,为了获得清晰可靠的拍摄效果,本发明的面阵相机21拍摄的图像在发送至图像处理单元3之前,分别经过第一光电转换元件22、光纤滑环23以及第二光电转换元件24进行光电转换,使得图像处理单元3接收的图像电信号趋于完整和可靠。The image acquisition unit 2 includes an area array camera 21 , a first photoelectric conversion element 22 , a fiber slip ring 23 and a second photoelectric conversion element 24 that are connected by communication in sequence. When the workpiece to be tested moves to a preset position on the detection station 11 When the detection position is close to the center of the shooting field of view of the area array camera 21, the proximity switch 12 is triggered and a trigger signal is sent to the computer control unit 6; the axis of the photosensitive lens of the area array camera 21 is set perpendicular to the flow direction of the detection station, taking into account the intelligent manufacturing workshop In the process, the operation of the equipment may cause certain vibrations at the detection station, thereby affecting the shooting accuracy of the area array camera 21 fixedly connected to the detection station. In order to obtain clear and reliable shooting effects, the area array camera 21 of the present invention shoots Before the image is sent to the image processing unit 3, it undergoes photoelectric conversion through the first photoelectric conversion element 22, the optical fiber slip ring 23 and the second photoelectric conversion element 24, so that the image electrical signal received by the image processing unit 3 tends to be complete and reliable.

具体地,第一光电转换元件22和第二光电转换元件24均设置有相同数量的输入端和多个输出端,光纤滑环的数量与之对应,第一光电转换元件22的每一个输入端分别与面阵相机连接,每一个输出端分别通过一个光纤滑环23与第二光电转换元件24的输入端连接,第二光电转换元件24的每一个输出端均与图像处理单元3连接。Specifically, the first photoelectric conversion element 22 and the second photoelectric conversion element 24 are both provided with the same number of input terminals and multiple output terminals, and the number of optical fiber slip rings corresponds thereto. Each input terminal of the first photoelectric conversion element 22 They are respectively connected to the area scan camera, each output end is connected to the input end of the second photoelectric conversion element 24 through an optical fiber slip ring 23, and each output end of the second photoelectric conversion element 24 is connected to the image processing unit 3.

图像处理单元3接收到经过光电转换的图像后,为了提高图像处理效率,本发明首先通过分辨率扫描获得当前检测工位的敏感区域图像并进行图像分割,然后仅仅对获得的敏感区域进行去噪以及边缘检测,传统的图像处理,都是对整个图像进行去噪再进行图像分割处理,本发明首先分割出敏感区域,再进行局部去噪,可以有效的提高图像处理效率。After the image processing unit 3 receives the photoelectrically converted image, in order to improve the image processing efficiency, the present invention first obtains the sensitive area image of the current detection station through resolution scanning and performs image segmentation, and then only denoises the obtained sensitive area. As well as edge detection, traditional image processing is to denoise the entire image and then perform image segmentation processing. The present invention first segments sensitive areas and then performs local denoising, which can effectively improve image processing efficiency.

具体地,图像处理单元3自动定位出图像上的多个标记点的中心位置,确定出背景标板14与水平方向的夹角,从而计算出面阵相机21与背景标板14之间的偏转角度,图像处理单元3控制该图像沿着偏转角度进行分辨率扫描,从而完成对当前检测工位的敏感区域的识别,识别出该敏感区域后,图像处理单元3对该图像进行图像分割操作,从而获得新的待处理图像,在优选的实施例中,考虑到背景标板14以及LED面光源矩阵15构成的背光照明环境,图像的敏感区域与背景的对比度较高,灰度值相差较大,本发明采用基于区域的图像分割算法分割出该敏感区域图像。Specifically, the image processing unit 3 automatically locates the center positions of multiple mark points on the image, determines the angle between the background target 14 and the horizontal direction, and thereby calculates the deflection angle between the area array camera 21 and the background target 14 , the image processing unit 3 controls the image to perform resolution scanning along the deflection angle, thereby completing the identification of the sensitive area of the current detection station. After identifying the sensitive area, the image processing unit 3 performs an image segmentation operation on the image, thereby Obtain a new image to be processed. In the preferred embodiment, considering the backlight lighting environment composed of the background target plate 14 and the LED surface light source matrix 15, the contrast between the sensitive area of the image and the background is high, and the gray value difference is large. The present invention uses a region-based image segmentation algorithm to segment the sensitive area image.

在对分割出的敏感区域图像进行去噪时,为了使得该敏感区域图像有更自然的平滑效果,增强对敏感区域图像的随机噪声的处理效果,本发明采用高斯滤波法对该敏感区域图像进行去噪;在获得平滑的敏感区域图像后,特征向量提取单元对处理后的敏感区域图像作进一步的处理。When denoising the segmented sensitive area image, in order to make the sensitive area image have a more natural smoothing effect and enhance the processing effect of random noise in the sensitive area image, the present invention uses the Gaussian filtering method to denoise the sensitive area image. Denoising; after obtaining a smooth sensitive area image, the feature vector extraction unit further processes the processed sensitive area image.

具体地,特征向量提取单元的目的是将以像素集合为特征的高维图像信息降为以向量集合为特征的低维图像信息,以便于计算机的处理以及保证深度神经网络单元分类的准确性;在本发明中,基于智能制造的柔性化生产特点,采用形状特征这一核心图像特征来获取待测工件的智能制造信息,是比较合适的,为了全面捕获待测工件敏感区域图像的制造信息,本发明同时考虑目标区域的外部边缘信息以及内部区域信息,将目标区域的边缘面积、边缘形状因子以及目标区域平均半径以及前3维的Hu不变矩作为表征敏感区域图像的特征变量,并以此作为敏感区域图像的特征向量,输入至基于深度神经网络单元5。Specifically, the purpose of the feature vector extraction unit is to reduce high-dimensional image information characterized by pixel sets into low-dimensional image information characterized by vector sets, so as to facilitate computer processing and ensure the accuracy of deep neural network unit classification; In the present invention, based on the flexible production characteristics of intelligent manufacturing, it is more appropriate to use the core image feature of shape characteristics to obtain the intelligent manufacturing information of the workpiece to be tested. In order to comprehensively capture the manufacturing information of the sensitive area image of the workpiece to be tested, This invention simultaneously considers the external edge information and internal area information of the target area, and uses the edge area, edge shape factor, average radius of the target area and the Hu invariant moment of the first three dimensions as characteristic variables to characterize the sensitive area image, and uses This is used as the feature vector of the sensitive area image and is input to the deep neural network-based unit 5.

在优选的实施例中,基于敏感区域图像的边缘属性,特征向量提取单元首先对敏感区域图像进行边缘检测,获得目标区域,并分别通过公式(1)-(3)计算获得目标区域的边缘面积、边缘形状因子以及目标区域平均半径,再加上前3维的Hu不变矩,构成具有四个特征变量的敏感区域的特征向量,以反映当前产品检测平台的加工或装配等制造质量信息,特征向量作为输入层发送至深度神经网络单元5;In a preferred embodiment, based on the edge attributes of the sensitive area image, the feature vector extraction unit first performs edge detection on the sensitive area image to obtain the target area, and calculates the edge area of the target area through formulas (1)-(3) respectively. , edge shape factor and average radius of the target area, plus the Hu invariant moments of the first three dimensions, form a feature vector of the sensitive area with four feature variables to reflect manufacturing quality information such as processing or assembly of the current product inspection platform. The feature vector is sent to the deep neural network unit 5 as an input layer;

上式中,参数M和N为目标区域的边缘点个数,其中t(x,y)为各边缘点的灰度值;参数L为目标区域的周长,可采用图像处理技术中的链码法进行计算获得,参考K为目标区域边界上的边缘点个数,(xk,yk)表示位于目标区域边界上的像素坐标,/>表示目标区域的质心坐标,可通过如下公式进行计算:In the above formula, the parameters M and N are the number of edge points in the target area, Among them, t(x,y) is the gray value of each edge point; the parameter L is the perimeter of the target area, which can be calculated using the chain code method in image processing technology. The reference K is the number of edge points on the boundary of the target area. Number, (x k ,y k ) represents the pixel coordinates located on the boundary of the target area,/> Represents the centroid coordinates of the target area, which can be calculated by the following formula:

其中,参数A表示敏感区域的面积,可在图像处理中识别到敏感区域时获得其大小。Among them, parameter A represents the area of the sensitive area, and its size can be obtained when the sensitive area is identified in image processing.

另外,Hu不变矩作为图像重要的全局特征,不受光线、噪声的影响,具有良好的几何不变形,可以有效地描述形状较为复杂的物体图像,考虑到智能制造典型的制造缺陷性质以及图像处理的效率,选取前3维Hu不变矩作为待测工件敏感区域图像的特征变量之一,是行之有效的,具体的计算方式,可参照图像处理技术中的常规做法,在此不一一赘述。In addition, Hu invariant moments, as an important global feature of images, are not affected by light and noise, have good geometric invariance, and can effectively describe images of objects with complex shapes. Taking into account the typical manufacturing defects and image properties of smart manufacturing, For processing efficiency, it is effective to select the first 3-dimensional Hu invariant moment as one of the characteristic variables of the sensitive area image of the workpiece to be tested. The specific calculation method can refer to the conventional practice in image processing technology, which is different here. A further explanation.

在优选的实施例中,本发明的深度神经网络单元5,具体是基于深度神经网络的制造缺陷预测模型,并包括三层神经网络,分别是输入层、隐含层和输出层,其中输入层与输出层具有相同的规模,输入层作为制造缺陷预测模型的输入接口,接收待测工件图像的特征向量,通过信息编码,到达隐含层,再经过信息解码变换到输出层,本发明采用经典的编码和解码公式模型,在此不一一赘述;在深度神经网络单元5正式进行待测工件的制造缺陷分类前,需要先进行学习训练,具体地,针对当前检测工位待测工件可能出现的典型制造缺陷,建立标准样本图像库,在优选的实施例中,标准样本图像库可以包括制造合格、制造缺陷I、制造缺陷II、制造缺陷III等四种样本库类型,以此作为深度神经网络的训练样本库;类似于前述的提取待测工件特征向量,本发明将标准样本图像库中的图像同样进行边缘检测,并依次提取图像的边缘面积、边缘标准差、形状因子以及Hu不变矩等特征变量,构成训练样本库的特征向量;最后,深度神经网络单元5的输入层读取训练样本库中的特征向量,基于深度神经网络单元5的编码与解码,对每个标准样本图像库中的图像对应的制造信息进行深度学习,从而获得当前检测工位的制造缺陷预测模型。In a preferred embodiment, the deep neural network unit 5 of the present invention is specifically a manufacturing defect prediction model based on a deep neural network, and includes a three-layer neural network, namely an input layer, a hidden layer and an output layer, where the input layer It has the same scale as the output layer. As the input interface of the manufacturing defect prediction model, the input layer receives the feature vector of the workpiece image to be tested, reaches the hidden layer through information encoding, and then transforms to the output layer through information decoding. The present invention adopts the classic The encoding and decoding formula models will not be described one by one here; before the deep neural network unit 5 can formally classify the manufacturing defects of the workpiece to be tested, learning and training need to be carried out first. Specifically, for the current inspection station, the workpiece to be tested may appear Typical manufacturing defects, establish a standard sample image library. In the preferred embodiment, the standard sample image library can include four sample library types: manufacturing qualified, manufacturing defect I, manufacturing defect II, manufacturing defect III, etc., as a deep neural network The training sample library of the network; similar to the aforementioned extraction of feature vectors of the workpiece to be tested, the present invention also performs edge detection on the images in the standard sample image library, and sequentially extracts the edge area, edge standard deviation, shape factor and Hu invariant of the image Feature variables such as moments constitute the feature vectors of the training sample library; finally, the input layer of the deep neural network unit 5 reads the feature vectors in the training sample library, and based on the encoding and decoding of the deep neural network unit 5, each standard sample image The manufacturing information corresponding to the images in the library is subjected to deep learning to obtain the manufacturing defect prediction model of the current inspection station.

深度神经网络单元5对训练样本库进行学习训练后,即可用于当前检测工位11的待测工件进行制造缺陷信息的分类预测,从而识别出当前产品检测平台1上的待测工件的制造缺陷类型,并将待测工件的制造缺陷信息进一步发送给计算机控制单元6,由计算机控制单元6进行制造缺陷的维修与处理。After the deep neural network unit 5 learns and trains the training sample library, it can be used to classify and predict the manufacturing defect information of the workpiece to be tested at the current inspection station 11, thereby identifying the manufacturing defects of the workpiece to be tested on the current product inspection platform 1 type, and further sends the manufacturing defect information of the workpiece to be tested to the computer control unit 6, and the computer control unit 6 performs repair and processing of the manufacturing defects.

在优选的实施例中,计算机控制单元6包括PLC控制器61,并与产品检测平台1、图像采集单元2和深度神经网络单元5分别通信连接,计算机控制单元6根据深度神经网络单元5的分类结果所指示的制造缺陷信息,结合历史维修数据库,确定出维修该制造缺陷所需的资源,并针对该缺陷类型、缺陷位置、缺陷程度、维护人员选择等方面自动生成维修该缺陷的维修策略,并形成相应的工作指令发送给PLC控制器61,由PLC控制器61执行具体的维修操作,并在历史维修数据库中更新添加一条针对该执行情况的维修记录。In a preferred embodiment, the computer control unit 6 includes a PLC controller 61 and is communicatively connected to the product detection platform 1, the image acquisition unit 2 and the deep neural network unit 5. The computer control unit 6 is classified according to the classification of the deep neural network unit 5. The manufacturing defect information indicated by the results is combined with the historical maintenance database to determine the resources required to repair the manufacturing defect, and automatically generate a maintenance strategy to repair the defect based on the defect type, defect location, defect degree, maintenance personnel selection, etc. Corresponding work instructions are formed and sent to the PLC controller 61, which performs specific maintenance operations, and updates and adds a maintenance record for the execution in the historical maintenance database.

实施方式二Embodiment 2

本发明进一步提供了一种应用于智能制造车间的故障在线检测方法,其采用本发明的前述智能制造车间故障在线检测系统,并包括如下步骤:The present invention further provides an online fault detection method applied to an intelligent manufacturing workshop, which adopts the aforementioned intelligent manufacturing workshop fault online detection system of the present invention and includes the following steps:

步骤1:构建基于深度神经网络的制造缺陷预测模型,通过样本图像对深度神经网络进行训练学习;Step 1: Construct a manufacturing defect prediction model based on a deep neural network, and train the deep neural network through sample images;

步骤2:智能定位与放行机构在计算机控制模块的控制指令下带动待测工件在检测工位上运动使其到达预设的检测位置,触发接近开关,并发送触发信号至计算机控制单元;Step 2: Under the control instructions of the computer control module, the intelligent positioning and release mechanism drives the workpiece to be tested to move on the detection station to reach the preset detection position, triggers the proximity switch, and sends the trigger signal to the computer control unit;

步骤3:计算机控制单元分别向LED面光源矩阵和图像采集单元发送指令,LED面光源矩阵打开照明,根据预先设定的程序和参数,图像采集单元对待测工件进行拍摄,光电转换后,将生成的图像发送至图像处理单元;Step 3: The computer control unit sends instructions to the LED surface light source matrix and the image acquisition unit respectively. The LED surface light source matrix turns on the lighting. According to the preset program and parameters, the image acquisition unit shoots the workpiece to be tested. After photoelectric conversion, it will generate The image is sent to the image processing unit;

步骤4:图像处理单元识别并分割出当前检测工位的敏感区域,针对该局部区域进行图像去噪处理,去噪后将该敏感区域图像发送至特征向量提取单元;Step 4: The image processing unit identifies and segments the sensitive area of the current detection station, performs image denoising on the local area, and sends the image of the sensitive area to the feature vector extraction unit after denoising;

步骤5:特征向量提取单元对敏感区域进行边缘检测,形成目标区域,并分别通过公式(1)-(3)计算获得目标区域的边缘面积、边缘形状因子以及目标区域平均半径,结合前3维的Hu不变矩,构成具有四个特征变量的敏感区域的特征向量;Step 5: The feature vector extraction unit performs edge detection on the sensitive area to form a target area, and calculates the edge area, edge shape factor and average radius of the target area through formulas (1)-(3) respectively, combined with the first three dimensions The Hu invariant moment constitutes the eigenvector of the sensitive area with four eigenvariables;

步骤6:基于训练好的深度神经网络对特征向量进行制造信息的诊断,预测并分类待测工件的制造缺陷,并将分类结果反馈给计算机控制单元;Step 6: Diagnose the manufacturing information based on the trained deep neural network on the feature vector, predict and classify the manufacturing defects of the workpiece to be tested, and feed the classification results back to the computer control unit;

步骤7:计算机控制单元基于分类结果以及历史维修数据库信息,确定出维修该制造缺陷的维修策略,发送工作指令给PLC控制器,由PLC控制器执行具体的维修操作。Step 7: Based on the classification results and historical maintenance database information, the computer control unit determines the maintenance strategy to repair the manufacturing defect, sends work instructions to the PLC controller, and the PLC controller performs specific maintenance operations.

在优选的实施例中,上述步骤1具体包括:In a preferred embodiment, the above step 1 specifically includes:

步骤1.1:构建基于深度神经网络的制造缺陷预测模型,所述深度神经网络为3层的神经网络,包括输入层、输出层、隐含层,其中输入层与输出层具有相同的规模;Step 1.1: Construct a manufacturing defect prediction model based on a deep neural network. The deep neural network is a three-layer neural network, including an input layer, an output layer, and a hidden layer, where the input layer and the output layer have the same scale;

步骤1.2:针对当前检测工位可能出现的典型制造缺陷,建立标准样本图像库,包括制造合格、制造缺陷I、制造缺陷II、制造缺陷III等四种样本库类型,作为深度神经网络的训练样本库;Step 1.2: Establish a standard sample image library for typical manufacturing defects that may occur at the current inspection station, including four sample library types: manufacturing qualified, manufacturing defect I, manufacturing defect II, and manufacturing defect III, as training samples for deep neural networks library;

步骤1.3:将标准样本图像库中的图像进行边缘检测,并依次提取图像的边缘面积、边缘标准差、形状因子以及Hu不变矩等特征变量,构成训练样本库的特征向量;Step 1.3: Perform edge detection on the images in the standard sample image library, and sequentially extract feature variables such as edge area, edge standard deviation, shape factor, and Hu invariant moment of the image to form the feature vector of the training sample library;

步骤1.4:深度学习网络单元的输入层读取所述训练样本库中的特征向量,并对每个标准样本图像库中的图像对应的制造信息进行深度学习,从而获得当前检测工位的制造缺陷预测模型。Step 1.4: The input layer of the deep learning network unit reads the feature vectors in the training sample library and performs deep learning on the manufacturing information corresponding to the images in each standard sample image library to obtain the manufacturing defects of the current inspection station. Predictive model.

以上述依据本发明的理想实施例为启示,通过上述的说明内容,相关工作人员完全可以在不偏离本项发明技术思想的范围内,进行多样的变更以及修改。本项发明的技术性范围并不局限于说明书上的内容,必须要根据权利要求范围来确定其技术性范围。Taking the above-mentioned ideal embodiments of the present invention as inspiration and through the above description, relevant workers can make various changes and modifications without departing from the scope of the technical idea of the present invention. The technical scope of the present invention is not limited to the content in the description, and must be determined based on the scope of the claims.

Claims (7)

1.一种故障在线检测系统,其特征在于,包括工件检测平台、图像采集单元、图像处理单元、特征向量提取单元、深度神经网络单元以及计算机控制单元,其中,1. An online fault detection system, characterized by including a workpiece detection platform, an image acquisition unit, an image processing unit, a feature vector extraction unit, a deep neural network unit and a computer control unit, wherein, 所述工件检测平台包括检测工位,所述图像采集单元内的面阵相机采集检测工位上工件检测平台的图像,并发送至图像处理单元;The workpiece detection platform includes a detection station, and the area array camera in the image acquisition unit collects images of the workpiece detection platform on the detection station and sends them to the image processing unit; 所述图像处理单元对接收到的图像进行分辨率扫描,获得当前检测工位的敏感区域图像,且对敏感区域图像进行去噪,再将去噪后的敏感区域图像发送至特征向量提取单元;The image processing unit performs a resolution scan on the received image to obtain the sensitive area image of the current detection station, denoises the sensitive area image, and then sends the denoised sensitive area image to the feature vector extraction unit; 特征向量提取单元对敏感区域图像进行边缘检测,形成目标区域,并分别通过公式(1)至(3)计算获得目标区域的边缘面积、边缘形状因子以及目标区域平均半径,再加上前3维的Hu不变矩,构成具有四个特征变量的敏感区域的特征向量,以反映当前工件检测平台的工件质量信息,特征向量作为输入层发送至深度神经网络单元;The feature vector extraction unit performs edge detection on the sensitive area image to form a target area, and calculates the edge area, edge shape factor and average radius of the target area through formulas (1) to (3) respectively, plus the first 3 dimensions The Hu invariant moments form a feature vector of the sensitive area with four feature variables to reflect the workpiece quality information of the current workpiece detection platform. The feature vector is sent to the deep neural network unit as an input layer; 上式中,参数M和N为目标区域的边缘点个数,其中t(x,y)为各边缘点的灰度值;参数L为目标区域的周长,采用图像处理技术中的链码法进行计算获得,参考K为目标区域边界上的边缘点个数,(xk,yk)表示位于目标区域边界上的像素坐标,表示目标区域的质心坐标,通过如下公式进行计算:In the above formula, the parameters M and N are the number of edge points in the target area, Among them, t(x,y) is the gray value of each edge point; the parameter L is the perimeter of the target area, which is calculated using the chain code method in image processing technology. The reference K is the number of edge points on the boundary of the target area. , (x k ,y k ) represents the pixel coordinates located on the boundary of the target area, Represents the centroid coordinates of the target area, calculated by the following formula: 其中,参数A表示敏感区域的面积,且适于在图像处理中识别到敏感区域时获得其大小;Among them, parameter A represents the area of the sensitive area, and is suitable for obtaining its size when a sensitive area is identified in image processing; 深度神经网络单元基于神经网络算法构建制造缺陷预测模型,对工件检测平台的图像特征向量进行训练、学习与分类,识别当前工件检测平台上的待测工件的制造缺陷类型,并将分类结果反馈给计算机控制单元;The deep neural network unit builds a manufacturing defect prediction model based on the neural network algorithm, trains, learns and classifies the image feature vectors of the workpiece detection platform, identifies the manufacturing defect type of the workpiece to be tested on the current workpiece detection platform, and feeds the classification results back to computer control unit; 所述工件检测平台还包括接近开关、智能定位与放行机构,在检测工位的一侧设置有沿着柔性生产线连续布置的背景标板,背景标板上设置有多个对应当前检测工位的标记点,检测工位的另一侧设置有LED面光源矩阵,该LED面光源矩阵与计算机控制单元通信连接,背景标板与LED面光源矩阵共同构成背光照明环境;以及The workpiece inspection platform also includes a proximity switch, an intelligent positioning and release mechanism, and a background mark plate continuously arranged along the flexible production line on one side of the inspection station. A plurality of background markers corresponding to the current inspection station are provided on the background mark plate. At the marking point, an LED surface light source matrix is provided on the other side of the detection station. The LED surface light source matrix is communicatively connected to the computer control unit. The background mark plate and the LED surface light source matrix together form a backlight lighting environment; and 所述工件检测平台的接近开关、智能定位与放行机构均与计算机控制单元通信连接,其中所述智能定位与放行机构接收计算机控制单元的控制指令,带动待测工件在检测工位上运动使其到达预设的检测位置并接近面阵相机的拍摄视野中心,触发接近开关并发送触发信号至所述计算机控制单元,由所述计算机控制单元启动所述图像采集单元进行图像采集;The proximity switch, intelligent positioning and release mechanism of the workpiece detection platform are all connected by communication with the computer control unit. The intelligent positioning and release mechanism receives the control instructions from the computer control unit and drives the workpiece to be tested to move on the detection station. Arrive at the preset detection position and approach the center of the shooting field of view of the area scan camera, trigger the proximity switch and send a trigger signal to the computer control unit, and the computer control unit starts the image acquisition unit to collect images; 所述图像处理单元自动定位出图像上的多个标记点的中心位置,确定出所述背景标板与水平方向的夹角,从而计算出所述面阵相机与所述背景标板之间的偏转角度,所述图像处理单元控制该图像沿着所述偏转角度进行分辨率扫描,从而完成对当前检测工位的敏感区域的识别。The image processing unit automatically locates the center positions of multiple mark points on the image, determines the angle between the background target and the horizontal direction, and thereby calculates the distance between the area array camera and the background target. Deflection angle, the image processing unit controls the image to perform resolution scanning along the deflection angle, thereby completing the identification of the sensitive area of the current detection station. 2.根据权利要求1所述的故障在线检测系统,其特征在于,2. The fault online detection system according to claim 1, characterized in that, 所述图像采集单元包括与面阵相机依次通信连接的第一光电转换元件、光纤滑环、第二光电转换元件,其中面阵相机的感光镜头轴线与检测工位流转方向垂直设置,面阵相机采集的图像经过光电转换后,由第二光电转换元件将图像的电信号发送至图像处理单元;The image acquisition unit includes a first photoelectric conversion element, an optical fiber slip ring, and a second photoelectric conversion element that are sequentially communicated with the area array camera. The axis of the photosensitive lens of the area array camera is perpendicular to the flow direction of the detection station. The area array camera After the collected image undergoes photoelectric conversion, the second photoelectric conversion element sends the electrical signal of the image to the image processing unit; 所述图像采集单元的第一光电转换元件和第二光电转换元件均设置有相同数量的输入端和多个输出端,所述光纤滑环的数量与之对应,所述第一光电转换元件的每一个输入端分别与所述面阵相机连接,所述第一光电转换元件的每一个输出端分别通过一个光纤滑环与所述第二光电转换元件的输入端连接,所述第二光电转换元件的每一个输出端均与所述图像处理单元连接。The first photoelectric conversion element and the second photoelectric conversion element of the image acquisition unit are both provided with the same number of input terminals and multiple output terminals, and the number of the optical fiber slip rings corresponds to the number of the first photoelectric conversion element. Each input end is connected to the area array camera respectively, and each output end of the first photoelectric conversion element is connected to the input end of the second photoelectric conversion element through an optical fiber slip ring. The second photoelectric conversion element Each output terminal of the component is connected to the image processing unit. 3.根据权利要求1所述的故障在线检测系统,其特征在于,3. The fault online detection system according to claim 1, characterized in that, 所述计算机控制单元包括PLC控制器,并与工件检测平台、图像采集单元和深度神经网络单元分别通信连接,所述计算机控制单元接收到工件检测平台的制造缺陷信息后,自动生成维修策略,并发出工作指令给PLC控制器,由PLC控制器执行具体的维修操作。The computer control unit includes a PLC controller and is communicated with the workpiece detection platform, the image acquisition unit and the deep neural network unit respectively. After receiving the manufacturing defect information of the workpiece detection platform, the computer control unit automatically generates a maintenance strategy and Send work instructions to the PLC controller, and the PLC controller will perform specific maintenance operations. 4.根据权利要求1-3任一项所述的故障在线检测系统,其特征在于,4. The fault online detection system according to any one of claims 1-3, characterized in that, 所述计算机控制单元根据所述深度神经网络单元的分类结果所指示的制造缺陷信息,结合历史维修数据库,确定出维修该制造缺陷所需的资源,并针对该缺陷类型、缺陷位置、缺陷程度、维护人员选择自动生成维修该缺陷的维修策略,并形成相应的工作指令发送给PLC控制器,由PLC控制器执行具体的维修操作,并在所述历史维修数据库中更新添加一条针对该执行情况的维修记录。The computer control unit determines the resources required to repair the manufacturing defect based on the manufacturing defect information indicated by the classification result of the deep neural network unit, combined with the historical maintenance database, and determines the defect type, defect location, defect degree, The maintenance personnel choose to automatically generate a maintenance strategy to repair the defect, and form a corresponding work instruction and send it to the PLC controller, which performs the specific maintenance operation and updates and adds an update to the historical maintenance database for the execution situation. Maintenance records. 5.采用如权利要求1所述的一种故障在线检测系统的应用于智能制造车间的故障在线检测方法,其特征在于,包括:5. An online fault detection method applied to an intelligent manufacturing workshop using a fault online detection system as claimed in claim 1, characterized in that it includes: 步骤1:构建基于深度神经网络的制造缺陷预测模型,通过样本图像对深度神经网络进行训练学习;Step 1: Construct a manufacturing defect prediction model based on a deep neural network, and train the deep neural network through sample images; 步骤2:工件检测平台在计算机控制模块的控制指令下带动待测工件在检测工位上运动使其到达预设的检测位置,触发接近开关,并发送触发信号至计算机控制单元;Step 2: Under the control instructions of the computer control module, the workpiece detection platform drives the workpiece to be tested to move on the detection station to reach the preset detection position, triggers the proximity switch, and sends the trigger signal to the computer control unit; 步骤3:计算机控制单元分别向LED面光源矩阵和图像采集单元发送指令,所述LED面光源矩阵打开照明,所述图像采集单元对待测工件进行拍摄,并将生成的图像发送至图像处理单元;Step 3: The computer control unit sends instructions to the LED surface light source matrix and the image acquisition unit respectively. The LED surface light source matrix turns on the illumination. The image acquisition unit photographs the workpiece to be tested and sends the generated image to the image processing unit; 步骤4:图像处理单元识别并分割出当前检测工位的敏感区域,针对该敏感区域进行图像去噪处理,去噪后将该敏感区域图像发送至特征向量提取单元;Step 4: The image processing unit identifies and segments the sensitive area of the current detection station, performs image denoising on the sensitive area, and sends the image of the sensitive area to the feature vector extraction unit after denoising; 步骤5:特征向量提取单元对敏感区域进行边缘检测,形成目标区域,并计算获得目标区域的边缘面积、边缘形状因子以及目标区域平均半径,结合前3维的Hu不变矩,构成具有四个特征变量的敏感区域的特征向量;Step 5: The feature vector extraction unit performs edge detection on the sensitive area to form a target area, and calculates the edge area, edge shape factor and average radius of the target area. Combined with the Hu invariant moments of the first three dimensions, it forms a four-dimensional Feature vector of the sensitive area of the feature variable; 步骤6:基于训练好的深度神经网络对特征向量进行制造信息的诊断,预测并分类待测工件的制造缺陷,并将分类结果反馈给计算机控制单元。Step 6: Diagnose the manufacturing information based on the trained deep neural network on the feature vector, predict and classify the manufacturing defects of the workpiece to be tested, and feed the classification results back to the computer control unit. 6.根据权利要求5所述的故障在线检测方法,其特征在于,6. The fault online detection method according to claim 5, characterized in that, 所述故障在线检测方法适于采用如权利要求1所述的故障在线检测系统检测工件的制造缺陷。The fault online detection method is suitable for detecting manufacturing defects of workpieces using the fault online detection system as claimed in claim 1. 7.根据权利要求6所述的故障在线检测方法,其特征在于,7. The fault online detection method according to claim 6, characterized in that, 所述步骤1:构建基于深度神经网络的制造缺陷预测模型,通过样本图像对深度神经网络进行训练学习的方法包括:Step 1: Construct a manufacturing defect prediction model based on a deep neural network. The method of training and learning the deep neural network through sample images includes: 步骤1.1:构建基于深度神经网络的制造缺陷预测模型,所述深度神经网络为3层的神经网络,包括输入层、输出层、隐含层,其中输入层与输出层具有相同的规模;Step 1.1: Construct a manufacturing defect prediction model based on a deep neural network. The deep neural network is a three-layer neural network, including an input layer, an output layer, and a hidden layer, where the input layer and the output layer have the same scale; 步骤1.2:针对当前检测工位可能出现的典型制造缺陷,建立标准样本图像库,包括制造合格、制造缺陷I、制造缺陷II、制造缺陷III四种样本库类型,作为深度神经网络的训练样本库;Step 1.2: Establish a standard sample image library for typical manufacturing defects that may occur at the current inspection station, including four sample library types: manufacturing qualified, manufacturing defect I, manufacturing defect II, and manufacturing defect III, as a training sample library for the deep neural network ; 步骤1.3:将标准样本图像库中的图像进行边缘检测,并依次提取图像的边缘面积、边缘标准差、形状因子以及Hu不变矩特征变量,构成训练样本库的特征向量;Step 1.3: Perform edge detection on the images in the standard sample image library, and sequentially extract the edge area, edge standard deviation, shape factor and Hu moment invariant feature variables of the image to form the feature vector of the training sample library; 步骤1.4:深度学习网络单元的输入层读取所述训练样本库中的特征向量,并对每个标准样本图像库中的图像对应的制造信息进行深度学习,从而获得当前检测工位的制造缺陷预测模型。Step 1.4: The input layer of the deep learning network unit reads the feature vectors in the training sample library and performs deep learning on the manufacturing information corresponding to the images in each standard sample image library to obtain the manufacturing defects of the current inspection station. Predictive model.
CN201811651018.2A 2018-12-31 2018-12-31 A fault online detection system and detection method applied to intelligent manufacturing workshops Active CN109840900B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811651018.2A CN109840900B (en) 2018-12-31 2018-12-31 A fault online detection system and detection method applied to intelligent manufacturing workshops

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811651018.2A CN109840900B (en) 2018-12-31 2018-12-31 A fault online detection system and detection method applied to intelligent manufacturing workshops

Publications (2)

Publication Number Publication Date
CN109840900A CN109840900A (en) 2019-06-04
CN109840900B true CN109840900B (en) 2023-12-19

Family

ID=66883679

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811651018.2A Active CN109840900B (en) 2018-12-31 2018-12-31 A fault online detection system and detection method applied to intelligent manufacturing workshops

Country Status (1)

Country Link
CN (1) CN109840900B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110989507A (en) * 2019-11-02 2020-04-10 温州大学 Automatic production digital twin workshop generation device that detects of circuit breaker
CN113217374B (en) * 2020-02-05 2023-05-30 中国石油天然气股份有限公司 Operation maintenance method and system for vertical well screw pump
CN111272775A (en) * 2020-02-24 2020-06-12 上海感图网络科技有限公司 Device and method for detecting defects of heat exchanger by using artificial intelligence
CN111583190B (en) * 2020-04-16 2022-07-22 浙江浙能技术研究院有限公司 Automatic identification method for hidden crack defect of internal cascade structure component
CN111721728B (en) * 2020-07-16 2023-02-21 陈皓 Fruit online detection device and use method thereof
CN112270284B (en) * 2020-11-06 2021-12-03 奥斯福集团有限公司 Lighting facility monitoring method and system and electronic equipment
CN113172422B (en) * 2021-05-18 2022-04-12 中品智能机械有限公司 Woodworking mechanical equipment assembly production line and assembly process thereof
CN113781448B (en) * 2021-09-14 2024-01-23 国电四子王旗光伏发电有限公司 Intelligent defect identification method for photovoltaic power station assembly based on infrared image analysis
CN113886627B (en) * 2021-10-09 2025-02-07 陕西通信规划设计研究院有限公司 A mobile communication system based on information synchronization
CN116108213B (en) * 2022-12-21 2024-10-25 东方晶源微电子科技(北京)股份有限公司 Method, device and equipment for establishing defect graph database and readable storage medium
CN116805204B (en) * 2023-08-24 2023-12-01 超网实业(成都)股份有限公司 Intelligent plant monitoring method and system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016201947A1 (en) * 2015-06-16 2016-12-22 华南理工大学 Method for automated detection of defects in cast wheel products

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018136262A1 (en) * 2017-01-20 2018-07-26 Aquifi, Inc. Systems and methods for defect detection

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016201947A1 (en) * 2015-06-16 2016-12-22 华南理工大学 Method for automated detection of defects in cast wheel products

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
"基于机器视觉的彩钢板缺陷检测和智能分类研究";孙创开;《中国优秀硕士学位论文全文数据库 信息科技辑》;20170715(第07期);第二章、第三章 *
"触点零件形貌在线自学习视觉检测系统研究";戴舒文;《中国优秀硕士学位论文全文数据库 信息科技辑》;20090915(第09期);第36-46页 *
基于BP神经网络的GIS缺陷图像识别系统的研究;万书亭等;《电力科学与工程》;20171128(第11期);全文 *

Also Published As

Publication number Publication date
CN109840900A (en) 2019-06-04

Similar Documents

Publication Publication Date Title
CN109840900B (en) A fault online detection system and detection method applied to intelligent manufacturing workshops
CN110543878B (en) Pointer instrument reading identification method based on neural network
CN111537517B (en) An Unmanned Intelligent Stamping Defect Identification Method
CN109693140B (en) An intelligent flexible production line and its working method
CN106952250B (en) Metal plate strip surface defect detection method and device based on fast R-CNN network
CN106226325B (en) A machine vision-based seat surface defect detection system and method
WO2022236876A1 (en) Cellophane defect recognition method, system and apparatus, and storage medium
CN102529019B (en) A method for mold detection, protection and parts detection and removal
CN108918527A (en) A kind of printed matter defect inspection method based on deep learning
CN111667455A (en) AI detection method for various defects of brush
CN108982514A (en) A kind of bionical vision detection system of casting surface defect
CN109978835B (en) Online assembly defect identification system and method thereof
JP2021515885A (en) Methods, devices, systems and programs for setting lighting conditions and storage media
CN112497219B (en) Columnar workpiece classifying and positioning method based on target detection and machine vision
CN107481244B (en) Manufacturing method of visual semantic segmentation database of industrial robot
CN112419237B (en) A method for surface defect detection of automobile clutch master cylinder groove based on deep learning
CN110186375A (en) Intelligent high-speed rail white body assemble welding feature detection device and detection method
CN115330734A (en) Automatic robot repair welding system based on three-dimensional target detection and point cloud defect completion
Yang et al. An automatic aperture detection system for LED cup based on machine vision
CN113592813B (en) New energy battery welding defect detection method based on deep learning semantic segmentation
CN114280075A (en) Online visual inspection system and method for surface defects of pipe parts
CN117314829A (en) Industrial part quality inspection method and system based on computer vision
CN112070712A (en) Printing defect detection method based on self-encoder network
CN118566235A (en) A method and system for visually detecting surface defects of fasteners
CN116843615B (en) An intelligent full inspection method for lead frames based on flexible optical paths

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 213100 No.28, Mingxin Middle Road, Wujin District, Changzhou City, Jiangsu Province

Applicant after: Changzhou Polytechnic

Address before: 213100 No.28, Mingxin Middle Road, Wujin District, Changzhou City, Jiangsu Province

Applicant before: Changzhou Institute of Industry Technology

GR01 Patent grant
GR01 Patent grant