KR102022496B1 - Process management and monitoring system using vision image detection and a method thereof - Google Patents

Abstract

The present invention relates to a process managing and monitoring system using vision image detection, and a method thereof. Specifically, the process managing and monitoring system using vision image detection, and the method thereof: receive and store an image and vision inspection data acquired through a plurality of machine vision devices; enable preparation of a report in real time by processing the stored data; output the acquired image and vision inspection result data in real time through the screen of a control center, and allow a manager to monitor the output image and data; and analyze the inspection result data in order to predict a facility of which check or verification is required according to the analysis result, to provide notifications thereof. Accordingly, the present invention replaces repetitive writing work having high possibility of mistakes, processes images of individual production facilities in order to analyze the size and direction of an edge, quickly senses a defect or deformation of the production facilities from the analysis results and thus, can improve productivity according to the state prediction of the production facilities.

Description

Process management and monitoring system using vision image detection and its method {PROCESS MANAGEMENT AND MONITORING SYSTEM USING VISION IMAGE DETECTION AND A METHOD THEREOF}

The present invention relates to a process management and monitoring system and method using vision image detection, and more particularly, to receive and store image and vision inspection result data acquired through a plurality of machine vision devices, and to process the stored data It is possible to prepare a report in real time, and output the acquired image and vision inspection result data in real time through the screen of the control center so that an administrator can monitor it, and analyze the inspection result data to check according to the analysis result Or by predicting and alerting you of equipment that needs confirmation, it replaces repetitive and error-prone handwriting work, and processes video images of each production facility to analyze the size and direction of the edges and from the results Production by quickly detecting defects or deformations The present invention relates to a process management and monitoring system and method using vision image detection, which can improve productivity according to the condition prediction of equipment.

In recent years, industrial automation has been accelerated by combining elements such as machine vision, robots, sensors, PLCs, computers, etc. across industries such as automobiles, electronics, displays, semiconductors, food and beverage, and sports. have.

As industrial automation is accelerating across industries, a variety of technologies are being introduced to automate the overhaul process that is performed before high volume products are shipped.

Vision inspection (machine vision inspection), one of such precision inspections, is performed by an image taken by a camera instead of a conventional method, which is judged by the human eye, for example, a labeling state of a product, a printing state of a wrapping paper, or a printing state of a barcode. To check the back.

Specifically, the vision inspection is performed by photographing the surface of the inspection object to obtain a surface image, and comparing the obtained surface image with a reference image to inspect the inspection object for defects.

The apparatus for vision inspection is installed in various places on the process line, and performs vision inspection on a product (hereinafter, inspection object) moving along the process line.

In other words, manufacturers form a large-scale production line, and each production line installs a number of vision equipment such as IP cameras to perform defect rate or quality inspection for each process.

On the other hand, in the field of modern manufacturing, image information by vision technology and automatic detection is more reliable and secure than any data. However, there is a lot of inconvenience, such as recording and pasting by hand, or re-recording with a digital camera to create a document because it can not process / edit the important information detected by the vision inspection device, and acts as a production deterrent. This can lead to workers not being able to focus on production and possibly to report something wrong by mistake.

In addition, since the defects of each process are manually linked to each other by document method, the material cost and labor cost increase, and it is difficult to extract the process data. Therefore, process data to be reported when requesting process data from external company or overseas market is manually recorded. You must write it.

In Korean Patent Publication [10-2009-0001710], it is possible to grasp the inventory history and work status of each detailed work line in real time and to understand the re-delivery of subcontractors to maximize manufacturing process efficiency and improve quality control. A manufacturing process management system is disclosed that enables the use of the same.

Korean Laid Open Patent [10-2009-0073643] discloses a production site information providing system and method capable of automatically aggregating production site information of a production facility factory line and providing it to an enterprise resource management system.

On the other hand, Korean Patent Registration [10-1570516] discloses an industrial vision integrated process management system.

On the other hand, the Korean Patent Registration [10-1920372] discloses a visual foresight system of equipment failure and poor quality of the automation line in the advanced product manufacturing plant optimized for small and medium-sized enterprises.

Korean public patent [10-2009-0001710] (published date: 2009. 01. 09) Korean public patent [10-2009-0073643] (published date: 2009. 07. 03) Korea Patent Registration [10-1570516] (Registration Date: November 13, 2015) Korea Patent Registration [10-1920372] (Registration Date: Nov. 14, 2018)

Accordingly, the present invention has been made to solve the above problems, an object of the present invention is to receive and store the image and vision inspection result data obtained through a plurality of machine vision devices, and process the stored data in real time It is possible to prepare a report, and output the acquired image and vision inspection result data in real time through the screen of the control center so that an administrator can monitor it, and analyze the inspection result data to check or check the result according to the analysis result. By predicting and alerting you of equipment that needs confirmation, it replaces repetitive and error-prone handwriting work, processes video images of each production facility, analyzes the size and orientation of the edges, and from the results, the production equipment defects. Or predict the state of a production facility by quickly detecting deformation Process control and monitoring system using a non-image detection to enhance the productivity of, and to provide the method.

The objects of the embodiments of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description. .

In the process management and monitoring system using a vision image detection according to an embodiment of the present invention for achieving the above object, a plurality of machine vision apparatus for performing a vision inspection of the inspection object to be processed in each work line (110, 120, 130); Receives and stores the video image and vision inspection result data from each machine vision device, automatically creates a document from the video image and the vision inspection result data, and the video image of a plurality of production facilities (510, 520, 530) Receives and stores and analyzes the edges by processing video images of the plurality of production facilities, and the damage, defect, and deformation of the equipment by linking the vision inspection result data of the inspection object and the image image analysis results of the production facility Central management server 200 for predicting the; A control center 300 for monitoring and controlling the process of each work line; And a terminal 400 for reading the created document.

The central management server 200, the interface unit 201 for interfacing to communicate with the plurality of machine vision devices, the plurality of production facilities, the control center, and the terminal; A processing unit 203 for performing various processes for process management and monitoring based on data transmitted and received through the interface unit; A database manager 204 for storing data necessary for process management and monitoring; A document automatic creation unit (205) for automatically creating a document from the image image and the vision inspection result data; To analyze the vision inspection result data during the preset period, determine process factors, defect cause analysis, and quality deviation analysis through inter-process linkages to identify the influence factors of defects or quality deviations, and to output facility status prediction data. An analysis unit 206; And a control unit 202 for controlling each component including the interface unit, the processing unit, the database management unit 204, the document automatic generation unit 205, and the analysis unit 206. .

The document automatic generation unit 205 may include an input / output unit 301 for receiving the video image and the vision inspection result data received from the machine vision apparatus and outputting a generated report; An image detector 302 for extracting a specific region from the video image; An image editing unit 303 for editing the detected image; A data processor 304 for processing the vision inspection result data received from the machine vision apparatus; And a report generator 305 for creating a report form document in real time from the edited image and the processed data.

Each machine vision device (110, 120, 130), the camera for shooting the inspection object (401); An illumination unit 406 for supplying light for surface image photographing of the inspection object; An image preprocessor 402 for preprocessing the image received from the camera; A vision inspection controller 403 for generating a surface image of the inspection object while controlling the operation of the camera and the lighting unit; A determination unit 404 for judging good or bad goods through the acquired inspection object image according to a predetermined vision inspection algorithm; And a communication unit 405 for transmitting the preprocessed image and the vision inspection result determined by the determination unit to the document automatic generation server.

The central management server may process image images of the plurality of production facilities to detect damages, defects, and deformations of the facility, and may include vision inspection result data of the inspection object, facility status prediction data, and the production facility. It characterized in that it further comprises a facility inspection unit 207 for predicting the abnormality of the facility in connection with the image image processing analysis results.

In addition, in the process management and monitoring method using a vision image detection according to an embodiment of the present invention, in the process management and monitoring method using a vision image detection, the central management of the image image of the inspection object photographed by the machine vision device Image transfer step (S1410) to deliver to the server; A vision inspection step (S1420) of determining, by the machine vision device, at least one director's vision inspection algorithm based on a predetermined vision inspection algorithm and outputting the defective or good product as vision inspection result data; A result delivery step of delivering, by the machine vision device, the vision inspection result data to the central management server (S1430); And a document creation step (S1440) in which the central management server automatically creates a report form document from the specific region extracted from the video image of the inspection object and the vision inspection result data.

The process management and monitoring method using the vision image detection, the monitoring step of outputting the video image of the inspection object on the process line, the vision inspection result data, and the video image of the production facilities in real time on the screen of the control center (S1460) It characterized in that it further comprises.

The process management and monitoring method using the vision image detection, after the document creation step (S1440), the edge management step (S1450) to analyze the edge by processing the image image of the production facility in the central management server (S1450); A prediction step (S1470) of analyzing a report generated during a predetermined period of time to identify a cause of failure or quality deterioration and output facility status prediction data; And a data linking step (S1480) of detecting a defect or deformation possibility of the production facility according to the edge analysis result and linking the facility state prediction data.

The process management and monitoring method using the vision image detection, after the data linking step (S1480), based on the equipment state prediction data and the edge analysis result, the alarm for outputting the warning and alarm for the equipment requiring action or inspection It further comprises the step (S1490).

Process management and monitoring method using the vision image detection, characterized in that it further comprises a test step for testing the performance of the vision inspection device and the automatic document creation performance.

According to the process management and monitoring system and the method using the vision image detection according to an embodiment of the present invention, the image and vision inspection result data obtained through a plurality of machine vision devices are received and stored, and the stored data is processed It is possible to prepare a report in real time, and output the acquired image and vision inspection result data in real time through the screen of the control center so that an administrator can monitor it, and analyze the inspection result data to check according to the analysis result Or by predicting and alerting you of equipment that needs confirmation, it replaces repetitive and error-prone handwriting work, and processes video images of each production facility to analyze the size and direction of the edges and from the results Quickly detect defects or deformations The productivity according to the state prediction can be improved.

In addition, according to the process management and monitoring system and the method using the vision image detection according to an embodiment of the present invention, the producer or line manager can concentrate on the production only without the need to make a report, it can occur when making a report Mistakes can be prevented.

In addition, according to the process management and monitoring system and the method using the vision image detection according to an embodiment of the present invention, through the image and inspection data stored in the database (storage device) can be remotely monitored in real time and real-time reporting have.

In addition, according to the process management and monitoring system and method using the vision image detection according to an embodiment of the present invention, by collecting and monitoring the data transmitted from the machine vision device to find a facility that affects the defect or quality, Analyzing quality deterrents has the effect of quality history management / tracking, cause analysis for major facilities that are found to be factors of quality, generation of predictive data for future facilities, and improvement of quality and output.

In addition, according to the process management and monitoring system and the method using the vision image detection according to an embodiment of the present invention, by processing the image image of each production facility in the central management server to quickly detect the defects and deformation of the production facility This has the effect of improving quality and yield.

1 is a block diagram of a process management and monitoring system using vision image detection according to an embodiment of the present invention.
Figure 2 is a detailed configuration diagram of one embodiment of the central management server of FIG.
3 is a detailed configuration diagram of an embodiment of an automatic document generator of FIG. 2.
4 is a detailed configuration diagram of one embodiment of the machine vision device of FIG.
5A and 5B illustrate an embodiment of a method for editing a vision image according to the present invention.
6 is a detailed configuration diagram of a vision inspection test apparatus for performance testing further included in the process management and monitoring system using vision image detection according to an embodiment of the present invention.
7A and 7B are diagrams illustrating an embodiment of a video image and a document stored according to an automatic document creation method in a management and monitoring system using vision image detection according to the present invention.
Figure 7c is an embodiment explanatory diagram of a document created by the document creating unit in the management and monitoring system using vision image detection according to the present invention.
8 is a diagram illustrating an embodiment of a screen output to a situation monitor of a control center in a management and monitoring system using vision image detection according to the present invention.
9 is a view for explaining a facility image image processing method for facility inspection in the management and monitoring system using vision image detection according to the present invention.
FIG. 10A is a mask according to an embodiment of the present invention used in processing a facility image for facility inspection in a management and monitoring system using vision image detection according to the present invention. FIG.
10B is an explanatory diagram of an image processed using the mask of FIG. 10A;
11A is a mask according to another embodiment for use in processing facility image images for facility inspection in a management and monitoring system using vision image detection in accordance with the present invention.
11B is an explanatory diagram of an image processed using the mask of FIG. 11A.
12A is a mask according to another embodiment for use in processing facility image images for facility inspection in a management and monitoring system using vision image detection in accordance with the present invention.
12B is an explanatory diagram of an image processed using the mask of FIG. 12A.
13A and 13B illustrate one embodiment for explaining before and after facility image image processing for facility inspection in a management and monitoring system using vision image detection according to the present invention;
14 is a flow diagram of one embodiment of a management and monitoring method using vision image detection in accordance with the present invention.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be.

On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "having" are intended to indicate that there is a feature, number, process, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present disclosure does not exclude the possibility of the presence or the addition of numbers, processes, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings will be described in more detail the present invention. Prior to this, terms or words used in the present specification and claims should not be construed as being limited to the ordinary or dictionary meanings, and the inventors should properly explain the concept of terms in order to best explain their own inventions. Based on the principle that it can be defined, it should be interpreted as meaning and concept corresponding to the technical idea of the present invention. In addition, unless there is another definition in the technical terms and scientific terms used, it has the meaning commonly understood by those of ordinary skill in the art to which this invention belongs, and the gist of the present invention in the following description and the accompanying drawings. Descriptions of well-known functions and configurations that may be unnecessarily blurred are omitted. The drawings introduced below are provided by way of example so that the spirit of the invention to those skilled in the art can fully convey. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. Also, like reference numerals denote like elements throughout the specification. It should be noted that the same elements in the figures are represented by the same numerals wherever possible.

1 is a block diagram of a process management and monitoring system using vision image detection according to an embodiment of the present invention.

As shown in FIG. 1, a process management and monitoring system using vision image detection according to an embodiment of the present invention includes a plurality of machine vision devices 110, 120, and 130, a central management server 200, and a control center ( 300, a terminal 400 and a number of production facilities 510, 520, 530.

Each of the machine vision devices 110, 120, and 130 is a device that performs machine vision inspection.

At this time, the machine vision inspection inspects the property of the object using one or more algorithms of pattern matching, point / line / face fitting, edge, color, and location. It is.

An application (ie, vision software) may be installed on the machine vision devices 110, 120, and 130 for machine vision inspection, and the application may acquire a target image of the inspection object and acquire the target image of the machine vision device 110 according to an algorithm. , 120 and 130 may be controlled.

In addition, the machine vision apparatuses 110, 120, and 130 may include a camera (not shown) for capturing an object, or may perform a machine vision inspection by communicating with a camera (not shown) located outside.

The machine vision apparatuses 110, 120, and 130 are interspersed in a plurality of process lines, and a plurality of machine vision apparatuses 110, 120, and 130 are installed, and transmit image information of real-time process states for each process state.

The machine vision apparatuses 110, 120, and 130 may have different APIs (application programming interfaces) and communication protocols supported by manufacturers, such as SDK, RTSP, HTTP CGI, and TCP / IP.

The central management server 200 receives and stores a video image and vision inspection result data from each of the machine vision apparatuses 110, 120, and 130, and automatically creates a document from the video image and the vision inspection result data. .

In addition, the central management server 200 receives and stores the video images of a plurality of production facilities (510, 520, 530), and processes the image images of the production facilities (510, 520, 530) damage of the equipment, Detect defects and deformations quickly. The central management server 200 may predict the abnormality of the facility by linking the analysis result of the vision inspection result data of the inspection object and the image image processing result of the production facilities 510, 520, and 530.

The control center 300 may monitor and control the plurality of machine vision devices 110, 120, 130 and the plurality of production facilities 510, 520, 530. The control center 300 is provided with a situation monitor for dividing and outputting the video images received from the plurality of machine vision devices (110, 120, 130) on one screen. In addition, in the situation monitor of the control center 300, the video images of the plurality of production facilities (510, 520, 530) may be divided and output on one screen.

The control center 300 is connected to the central management server 200 through a network to receive process state image information, extract event information according to the management policy from the process state image information, such as occurrence of abnormal situation or failure Notify the alarm for each event information, set the information about the machine vision apparatus (110, 120, 130) and the production equipment (510, 520, 530), history management function of the process state image information, manager management function, It performs configuration management function, performance management function, fault management function and information inquiry function, and performs control for each process based on process status image information.

The control center 300 provides a situation monitor (not shown) for setting the information and group of the machine vision apparatus 110 and displaying the situation according to the setting event to collectively manage the process state image information, and divides the screen. It provides integration, checking recorded and real-time video, bookmarks, preview and image storage.

The terminal 400 may read the automatically generated document (report), and may receive an alarm signal according to the analysis and the facility inspection of the central management server 200.

In this case, the terminal 400 may be a person in charge PC or board room PC for each process line, the monitoring information, the degree of participation of each process status or statistics and report creation or reading level is set according to the preset management grade.

For example, when the terminal 400 is the person in charge of the PC, process state image information is transmitted to monitor the process state of the corresponding process line, and statistics on the process state after checking the defective rate by analyzing the process state image information And reports.

On the other hand, when the terminal 400 is a boardroom PC, based on the statistics and reports received from the person in charge PC and the process state image information of each process line to monitor the current production rate and the defective rate for each process, the specific process through You can also send a work order to the person in charge.

On the other hand, the terminal 400 is at least one multimedia terminal including a mobile terminal such as a smart device such as a PC, a smart phone, a smart pad and a tablet PC, a mobile phone, a PDA, a notebook, and the like through a communication network vision image Connect with process control and monitoring system using detection.

Here, the communication network may be configured without regard to communication modes such as wired and wireless, and may include a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). It may be configured with a variety of communication networks. Preferably, the communication network referred to in the present invention may be a known World Wide Web (WWW).

Each production facility 510, 520, 530 is a component required for the process, and in the present invention, the image image of each production facility 510, 520, 530 is transmitted to the central management server 200.

In addition, the process management and monitoring system using vision image detection according to an embodiment of the present invention may further include a vision inspection test device 600.

The vision inspection test apparatus 600 may determine the performance of the machine vision apparatus and determine whether the automatic document creation is properly performed. The detailed configuration of the vision inspection test apparatus 600 will be further described with reference to FIG. 6.

In addition, the vision inspection test apparatus 600 may be used to determine a performance of processing video images of the plurality of production facilities 510, 520, and 530.

2 is a detailed block diagram of an embodiment of the central management server of FIG.

2, the central management server 200 according to the present invention includes an interface unit 201, a control unit 202, a processing unit 203, a database management unit 204, an automatic document generation unit 205, and an analysis. Section 206.

The interface unit 201, the control unit 202, the processing unit 203, the database management unit 204, the document automatic creation unit 205, and the analysis unit 206 are at least a part of the central management server. Program modules in communication with 200. Such program modules may be included in the central management server 200 in the form of operating systems, application modules, and other program modules, and may be physically stored on various known storage devices. In addition, these program modules may be stored in a remote storage device that can communicate with the central management server 200. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

The central management server 200 communicates with a plurality of machine vision devices (110, 120, 130), a plurality of production facilities (510, 520, 530), control center 300, and the terminal 400 through a communication network And transmit / receive data necessary for process management and monitoring to / from multiple machine vision devices (110, 120, 130), multiple production facilities (510, 520, 530).

On the other hand, the interface unit 201 is the central management server 200 a plurality of machine vision devices (110, 120, 130), a plurality of production facilities (510, 520, 530), control center 300, and the terminal Interface to perform communication with the 400, and may provide the terminal 400 with a graphical user interface necessary for the transmission and reception of data related to process management and monitoring.

The control unit 202 controls the flow of data between the interface unit 201 as described above, the processing unit 203, the database management unit 204, the document automatic creation unit 205, and the analysis unit 206 to be described later. Perform the function. That is, the control unit 202 controls each component including the interface unit 201, the processing unit 203, the database management unit 204, the document automatic creation unit 205, and the analysis unit 206.

The processing unit 203 performs various processes for process management and monitoring based on data transmitted and received through the interface unit 201. This processing according to the present invention will be described later.

The database manager 204 may include a facility database 204a for managing information on facilities required for factory automation, a check database 204b for storing and managing analysis and inspection results of the facilities, and each machine vision device. An original image database 204c storing an original image image transmitted from 110, 120, and 130, a detection image database 204d storing an image detected by the document automatic creation unit, and the document automatic creation unit It may include a document database (205e) and the like that stores the document created by the.

For example, the facility database 204a may include information about the name of the facility, the manufacturer, the facility ID, the control signal, the function, the location, and the like. The facility database 204a may be referred to by the processing unit 203 so that at least some of the information may be transmitted to the external device (terminal) through the interface unit 201.

The inspection database 204b may include information about a video image of the facility, an image of the edge processed image, and a check history, a repair history, and the like of the facility. This check database 204b may be referenced by the processing unit 203.

The original image database 204c may include an original image image received from a machine vision apparatus, a recording date, a time, and the like of the corresponding image image.

The detection image database 204d stores a detected image of a specific area in the original image image.

In the above embodiment, a database for storing the information for the implementation of the present invention includes the facility database 204a, the inspection database 204b, the original image database 204c, the detection image database 204d, and the document database 204e. Although classified into branch databases, the configuration of the database including such a classification can be changed according to the needs of those skilled in the art.

On the other hand, in the present invention, a database is a concept that includes not only a negotiated database but also a database of a broad meaning including a data record based on a computer file system, and the like. It should be understood that the data of the present invention can be included in the database of the present invention if it can be extracted.

The document creating unit 205 automatically creates a document from the video image and the vision inspection result data.

The analysis unit 206 analyzes the vision inspection result data during the preset period, and performs process status determination, failure cause analysis, and quality deviation analysis through linkage between processes to identify the influence factors of the defects or quality deviations, and the equipment. Output state prediction data.

In addition, the central management server 200 processes image images of the plurality of production facilities (510, 520, 530) to quickly detect the damage, defect, and deformation of the facility, the vision inspection result data of the inspection object, the The facility inspection unit 207 further predicts whether there is an abnormality in the facility by linking the facility state prediction data and the analysis result of the video image processing of the production facilities 510, 520, and 530.

3 is a detailed configuration diagram of an embodiment of an automatic document generating unit of FIG. 2.

As shown in FIG. 3, the document creating unit 205 includes an input / output unit 301, an image detector 302, an image editing unit 303, a data processing unit 304, and a report generating unit 305.

The input / output unit 301 receives a video image and vision inspection result data received from the machine vision apparatuses 110, 120, and 130, and outputs a generated report.

The image detector 302 extracts a specific area from the video image.

As a method of extracting a specific region, a position of a predetermined reference point may be used in the photographed image of the inspection object. The predetermined reference point may be a portion printed or displayed on the surface of the inspection object.

The image editing unit 303 edits the detected image when it is necessary to edit it. For example, the image editing unit 303 may perform not only rotation but also editing such as shift and zoom on the detected image.

The data processor 304 processes the vision inspection result data.

The report generator 305 creates a report form document in real time from the detected image or the edited image and the data processed by the data processor 304.

4 is a detailed block diagram of an embodiment of the machine vision apparatus of FIG. 1.

As shown in FIG. 4, each machine vision apparatus includes a camera 401, an image preprocessor 402, a vision inspection control unit 403, a determination unit 404, a communication unit 405, and an illumination unit 406. .

The camera 401 photographs an inspection object and provides an image photographed to the image preprocessor 402 to be described later.

The illumination unit 406 emits illumination light for photographing the surface image, and emits light toward the inspection object while emitting illumination in front of the camera 401. The lighting unit 406 may be configured as a substrate on which a plurality of LED elements are mounted, and may be preferably disposed behind the camera 401.

The image preprocessor 402 preprocesses the image of the inspection object and stores the image in a storage unit (not shown), or the preprocessed image is stored in a separate storage device (eg, a central management server) through the communication unit 405. And a database manager 204 of 200.

The vision inspection control unit 403 is a component that generates the surface image of the inspection object while controlling the operation of the camera 401 and the lighting unit 406 described above.

The vision inspection control unit 403 controls the overall operation of the machine vision device, and may include a processor such as a CPU. The vision inspection controller 403 may control other components included in the machine vision apparatus to perform an operation corresponding to a user input received through an input / output unit (not shown).

For example, the vision inspection control unit 403 may execute a program stored in the memory, read a file stored in the memory, or store a new file in the memory.

Here, the vision inspection control unit 403 sets a photographing condition of the camera 401 (for example, an exposure value or a sensitivity of the camera 401, an aperture setting, a shutter speed, etc.) while generating the surface image of the inspection object. In addition, by generating a plurality of surface images, it is possible to generate a high quality image.

The determination unit 404 determines the good or bad goods through the acquired inspection object image according to a preset vision inspection algorithm, and stores the vision inspection result thereof in a storage unit (not shown), or the vision inspection The result is stored in a separate storage device (eg, the database manager 204 of the central management server 200) through the communication unit 405.

That is, the image acquired through the camera 401 is interpreted through vision software. Vision software can match patterns of objects from the extracted images, fit points, lines or faces, distinguish colors, gauge objects, or provide location information for robot guides. In addition, one-dimensional and two-dimensional barcodes displayed on the object may be read or optical character reading (OCR) may be performed. This process allows objects to be inspected, measured or read, enabling industrial automation.

5A and 5B illustrate an embodiment of a method of editing a vision image according to the present invention.

5A and 5B illustrate a method of editing images 510 and 520 received from the machine vision apparatuses 110, 120, and 130 by the image editing unit 303 of the automatic document generation unit 205 according to the present invention. .

In the first image 510, the inspection object 511 is correctly placed and photographed, and the distribution period region 512 may be extracted in a quadrangular shape.

On the other hand, in the second image 520, when the inspection object 521 is photographed without being placed correctly and the extraction period region 522 is extracted, an operation of rotating is required.

On the other hand, when the image sharpness of the region to be extracted is inferior, the image editing unit 303 may perform an operation of increasing the sharpness through filtering.

In addition, the image editing unit 303 may edit not only a rotation but also a shift and a zoom.

6 is a detailed configuration diagram of a vision inspection test apparatus for performance testing that is further included in a process management and monitoring system using vision image detection according to an embodiment of the present invention.

As illustrated in FIG. 6, the vision inspection test apparatus 600 includes a test image generator 601 and a simulation unit 602.

The test image generator 601 generates a virtual test image for simulating the operation of the machine vision apparatus 110, 120, 130, and the test image generator 601 receives an input from a user or a random image sample. Create a test image with different degrees of variation.

The degree of deformation can be divided into several stages, and the factors of deformation include brightness, noise, contrast, and shading, depending on vibration, camera positional deviation, poor focus, foreign objects and damage. ), Shift, rotation, lens flare, zoom, balance, blur, dust, scratch, and the like.

The simulation unit 602 simulates the operations of the machine vision devices 110, 120, and 130 using the virtual test image generated by the test image generation unit 601.

Each of the machine vision apparatuses 110, 120, and 130, the test image generator 601, and the simulation unit 602 may be included or included in each other. That is, for example, the test image generator 601 including the simulation unit 602 may be included in the machine vision devices 110, 120, and 130, or may generate a test image including the simulation unit 602. The unit 601 may include machine vision devices 110, 120, and 130, or the simulation unit 602 including the machine vision devices 110, 120, and 130 may include a test image generator 601. It may be located outside and communicate with the test image generator 601.

In addition, each of the machine vision devices 110, 120, and 130, the test image generator 601, and the simulation unit 602 may be located outside each other and communicate with each other through a network.

As illustrated in FIG. 6, the vision inspection test apparatus 600 including the test image generator 601 and the simulation unit 602 may be provided in a separate PC or may be driven under the operation of vision software.

The vision inspection test apparatus 600 may generate a test image generation number, a test image deformation degree setting, and a test simulation operation of the machine vision apparatus through a vision inspection application (vision software).

In addition, the vision inspection test apparatus 600 may test processing performance of image images of a plurality of production facilities 510, 520, and 530.

7A and 7B are diagrams illustrating an embodiment of a video image and a document stored according to an automatic document creation method in a management and monitoring system using vision image detection according to the present invention.

Figure 7a shows that the data received from the machine vision device is organized into folders by date according to the management and monitoring method using the vision image detection according to the present invention. Shows that there are folders and auto-created documents.

7C is a diagram illustrating an embodiment of a document created by a document preparing unit in a management and monitoring system using vision image detection according to the present invention.

As illustrated in FIG. 7C, a document prepared according to a document automatic creation method using vision image detection according to the present invention may be viewed in a product name 701, a start time 702, an end time 703, and an inspection count 704. , A good article 705, a defective 706, a defective rate 707, an edited image 708, and a link 709 of the entire image.

That is, in the management and monitoring system using vision image detection according to the present invention, it is possible to generate a report type document in real time by receiving a result of vision inspection.

8 is a diagram illustrating an embodiment of a screen output to a situation monitor of a control center in a management and monitoring system using vision image detection according to the present invention.

As shown in FIG. 8, a plurality of vision inspection results are divided and output in real time on one monitor screen. For example, an edited image, a production quantity, a defective rate, etc. from line 1 to line 9 are displayed.

On the other hand, the edge (Edge) refers to the portion where the brightness is sharply changed in the image, it means a boundary line or contour. By detecting the edge, information about the position, shape, size, etc. of the object can be known. Edge detection is the process of finding edge pixels, and edge enhancement is to increase the contrast between the edge and the background to make the edge more visible. Edge tracing is the process of following an edge.

An edge means a place where a large difference in pixel values occurs, such as a discontinuous region of pixel values, and thus, the detection of the edge uses a concept of differential operation for measuring spatial variation. That is, since the edge is the rate of change of contrast or brightness based on the contrast of the image, that is, the slope. The slope in the image is called a gradient, and you can get the edge by getting the gradient.

Gradient is an extension of the first derivative to two dimensions (x, y) and is a measure of the measure of variation in the function.

The edge detection methods include a detection method using a first derivative and a detection method using a second derivative. The detection method using the second derivative is a method of improving the performance of edge detection by adding the derivative once more to the result obtained as the first derivative.

9 is a view illustrating a facility image image processing method for facility inspection in a management and monitoring system using vision image detection according to the present invention.

As shown in FIG. 9, the processing of video images for edge detection for production facilities uses an image processing convolution technique.

It is called spatial area based processing or mask based processing that performs operations based on a prescribed area called a mask, not pixel based processing that performs various operations on each pixel value.

Mask-based processing calculates the output pixel value 904 by correspondingly multiplying 903 the element values in the mask 902 and the pixel values 901 of the input image 900 in the spatial region. Performing this process while moving for all input pixel values is called a line (convolution). In this case, a mask multiplied by the input image is called a kernel, a window, a filter, or the like.

That is, the new output pixel value 904 is a value obtained by multiplying the pixel values 901 of the input image by the weights of the corresponding line masks 902 respectively (903).

 Examples of first order derivatives include the Robert operator, the Sobel operator, and the Prewitt operator. This will be described with reference to FIGS. 10A to 12B.

FIG. 10A is a mask according to an exemplary embodiment used in processing a facility image for facility inspection in a management and monitoring system using vision image detection according to the present invention.

10A shows the row detection mask G _x and the column detection mask G _y of the Robert operator.

The Robert operator is smaller in size than other masks and can be used effectively, but has a disadvantage of being sensitive to noise because it is difficult to use the extruded values appropriately for the average.

10B is an explanatory diagram of an image processed using the mask of FIG. 10A.

Referring to FIG. 10B, there is an original image on the upper left, a horizontal image using a row detection mask on the upper right, a vertical image using a column detection mask on the lower left, and a row detection mask and a column detection mask on the lower right. The image using both is shown.

FIG. 11A is a mask according to another exemplary embodiment used in processing a facility image for facility inspection in a management and monitoring system using vision image detection according to the present invention.

11A illustrates a row detection mask G _x and a column detection mask G _y of the Sobel operator.

 The Sobel operator has the advantage of further emphasizing the center value of the original image, and is efficient at averaging protrusion values, but sensitive to diagonal edges.

11B is an explanatory diagram of an image processed using the mask of FIG. 11A.

Referring to FIG. 11B, there is an original image on the upper left, a horizontal image using a row detection mask on the upper right, a vertical image using a column detection mask on the lower left, and a row detection mask and a column detection mask on the lower right. The image using both is shown.

12A is a mask according to another exemplary embodiment used in processing a facility image for facility inspection in a management and monitoring system using vision image detection according to the present invention.

12A illustrates a row detection mask G _x and a column detection mask G _y of a Prewitt operator.

The Prewitt operator has the same advantages as the Sobel filter, with slightly faster response times and relatively efficient averaging of the protruding values, but sensitive to horizontal and vertical edges, It is less pronounced.

12B is an explanatory diagram of an image processed using the mask of FIG. 12A.

Referring to FIG. 12B, there is an original image on the upper left, a horizontal image using a row detection mask on the upper right, a vertical image using a column detection mask on the lower left, and a row detection mask and a column detection mask on the lower right. The image using both is shown.

On the other hand, the size of the edge can be expressed as shown in [Equation 1].

[Equation 1]

Here, G (x, y) is an edge size, G _x is a row detection mask, and G _Y is a column detection mask.

On the other hand, the direction of the edge can be expressed as shown in [Equation 2].

[Equation 2]

Is the direction of the edge, G _x is the row detection mask, and G _Y is the column detection mask.

13A and 13B illustrate an exemplary embodiment for explaining before and after facility image image processing for facility inspection in a management and monitoring system using vision image detection according to the present invention.

FIG. 13A is an image before the facility inspection unit processes the image image of the production facility, and FIG. 13B is an image after the facility inspection unit processes the image image of the production facility using the Sobel mask.

Referring to FIGS. 13A and 13B, by processing video images of a production facility, edge detection is possible, and changes in damage, defects, and deformation of the production facility may be detected.

Therefore, in addition to the vision inspection results on the product (inspection object) in the process, as well as connected to each other through the detection of changes in the appearance of the production equipment, the failure prediction and defect prevention of the equipment can be made quickly, productivity can be improved.

14 is a flow diagram of an embodiment of a process management and monitoring method using vision image detection in accordance with the present invention.

First, the image of the inspection object photographed by the machine vision apparatus (110, 120, 130) is transmitted to the central management server 200 (S1410).

Thereafter, the machine vision apparatuses 110, 120, and 130 analyze the video image of the photographed inspection object according to a predetermined vision inspection algorithm, determine a defective or good product, and output the result as vision inspection result data (S1420).

Thereafter, the machine vision apparatuses 110, 120, and 130 transmit the vision inspection result data to the central management server 200 (S1430).

Here, the video image of the inspection target and the vision inspection result data may be transmitted to the central management server 200 together. The video image of the inspection object and the inspection result data are associated and stored in the database management unit 204 of the central management server 200.

Thereafter, the central management server 200 automatically creates a report format document from the specific region extracted from the video image of the inspection object and the vision inspection result data (S1440).

On the other hand, the central management server 200 analyzes the image image of the production facilities to analyze the size and direction of the edge (S1450).

Meanwhile, an image image of the inspection object on the process line, the vision inspection result data, and an image image of the production facilities are output in real time through the screen of the control center (S1460).

The step (S1460) of outputting through the screen of the control center may be performed in real time as the related image is received from the control center regardless of the order of steps.

Thereafter, the report generated during the predetermined period of time is analyzed to identify the cause of the defective or deteriorated quality and output the facility state prediction data (S1470).

Subsequently, a defect or a deformation possibility of a production facility is detected according to the edge analysis result, and it is associated with the facility state prediction data (S1480).

Subsequently, a warning and an alarm for a facility requiring action or inspection are output based on the facility state prediction data and the edge analysis result (S1490).

Although not shown in the drawings, the process management and monitoring method using the vision image detection according to the present invention further includes a test step to determine the performance of the machine vision device and to determine whether the automatic report generation is performed without a problem can do. In the test step, by generating a test image, and input it to the machine vision device to simulate, it is possible to know the performance of the machine vision device and the automatic document creation performance.

In addition, the process management and monitoring method using the vision image detection according to the present invention may further include a test step for determining the performance of the production facility inspection through the edge analysis. In the test step, by generating a test image, and input it to the central management server to simulate, it is possible to determine the performance of the production facility inspection through the edge analysis.

Although a process management and monitoring method using vision image detection according to an embodiment of the present invention has been described above, a computer-readable recording medium and a vision image storing a program for implementing the process management and monitoring method using vision image detection are described. The program stored in the computer-readable recording medium for implementing the process management and monitoring method using the detection can also be implemented.

That is, those skilled in the art can easily understand that the above-described process management and monitoring method using vision image detection may be provided in a recording medium that can be read through a computer by program of instructions for implementing the same. There will be. In other words, it may be embodied in the form of program instructions that can be executed by various computer means, and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in computer software. Examples of such computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and floptical disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, USB memory, and the like. The computer-readable recording medium may be a transmission medium such as an optical or metal wire, a waveguide, or the like including a carrier wave for transmitting a signal specifying a program command, a data structure, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The present invention is not limited to the above-described embodiments, and the scope of application is not limited, and various modifications can be made without departing from the gist of the present invention as claimed in the claims.

110, 120, 130: machine vision device 200: Central Management Server
300: control center 400: terminal
510, 520, 530: production plant 600: vision inspection test device
201: interface unit 202: control unit
203: processing unit 204: database management unit
205: document automatic creation unit 206: analysis unit
207: facility inspection unit 301: input and output unit
302: Image detector 303: Image editor
304: data processing unit 305: report creation unit
401: camera 402: image preprocessor
403: vision inspection control unit 404: determination unit
405: communication unit 406: lighting unit
601: test image generation unit 602: simulation unit

Claims (10)

In the process control and monitoring system using vision image detection,
A plurality of machine vision devices (110, 120, 130) for performing vision inspection of the inspection object processed in each work line;
Receives and stores the video image and vision inspection result data from each machine vision device, automatically creates a document from the video image and the vision inspection result data, and the video image of a plurality of production facilities (510, 520, 530) Receives and stores and analyzes the edges by processing video images of the plurality of production facilities, and the damage, defect, and deformation of the equipment by linking the vision inspection result data of the inspection object and the image image analysis results of the production facility Central management server 200 for predicting the;
A control center 300 for monitoring and controlling the process of each work line;
A terminal 400 for reading the created document; And
Vision inspection test apparatus 600 for testing the performance of the plurality of machine vision devices, the automatic creation of the document and the image image processing performance of the plurality of production facilities
Including,
The central management server 200,
An interface unit 201 for interfacing to communicate with the plurality of machine vision devices, the plurality of production facilities, the control center, and the terminal;
A processing unit 203 for performing various processes for process management and monitoring based on data transmitted and received through the interface unit;
A database manager 204 for storing data necessary for process management and monitoring;
A document automatic creation unit (205) for automatically creating a document from the image image and the vision inspection result data;
To analyze the vision inspection result data during the preset period, determine process factors, defect cause analysis, and quality deviation analysis through inter-process linkages to identify the influence factors of defects or quality deviations, and to output facility status prediction data. An analysis unit 206; And
Control unit 202 for controlling each component including the interface unit, the processing unit, the database management unit 204, the document automatic creation unit 205, and the analysis unit 206
Characterized in that it comprises a,
The document automatic creation unit 205,
An input / output unit (301) for receiving the video image received from the machine vision device and the vision inspection result data, and outputting a generated report;
An image detector 302 for extracting a specific region from the video image;
An image editing unit 303 for editing the detected image;
A data processor 304 for processing the vision inspection result data received from the machine vision apparatus; And
Report generating unit 305 for creating a report form document in real time from the edited image and the processed data
Characterized in that it comprises a,
The image detector 302,
Using a position of a predetermined reference point in the photographed video image of the inspection object, wherein the predetermined reference point is a portion that is printed or displayed on the surface of the inspection object,
The image editing unit 303,
Rotation, filtering, shift, and zoom editing are performed on the detected image.
Regarding video images of the plurality of production facilities,
The process of calculating the output pixel value 904 by correspondingly multiplying 903 the element values in the mask 902 and the pixel values 901 of the input image 900 in the spatial area for edge detection is performed on all input pixel values. Characterized by using an image processing line technique that is performed while moving about,
The size of the edge is represented by the following [Equation 3],
[Equation 3]

(Where G (x, y) is the size of the edge, G _x is the row detection mask, and G _y is the column detection mask),
The direction of the edge is characterized in that represented by the following [Equation 4],
[Equation 4]

(Where θ is the direction of the edge, G _x is the row detection mask, G _y is the column detection mask),
The vision inspection test device 600,
A test image generator 601 for generating a virtual test image by applying a different degree of deformation to a user's input or a random image sample; And
A simulation unit 602 for simulating the operation of the machine vision device using the virtual test image generated by the test image generator.
Process management and monitoring system using a vision image detection comprising a.
delete delete The method of claim 1,
Each machine vision device (110, 120, 130),
A camera 401 for photographing the inspection object;
An illumination unit 406 for supplying light for surface image photographing of the inspection object;
An image preprocessor 402 for preprocessing the image received from the camera;
A vision inspection controller 403 for generating a surface image of the inspection object while controlling the operation of the camera and the lighting unit;
A determination unit 404 for judging good or bad goods through the acquired inspection object image according to a predetermined vision inspection algorithm; And
The communication unit 405 for transmitting the pre-processed image and the vision inspection result determined by the determination unit to the document automatic creation server
Process management and monitoring system using a vision image detection, further comprising.
The method of claim 1,
The central management server,
Image images of the plurality of production facilities are processed to detect damage, defects, and deformation of the facility, and vision inspection result data of the inspection object, the facility state prediction data, and image image processing analysis results of the production facility. Facility inspection unit (207) for predicting the abnormality of the facility in connection with
Process management and monitoring system using a vision image detection, further comprising.
A process management and monitoring method using vision image detection in a process management and monitoring system using vision image detection,
An image transfer step (S1410) of transferring a video image of the inspection object photographed by the machine vision device to the central management server;
A vision inspection step (S1420) of determining, by the machine vision apparatus, a defective or good product according to at least one preset vision inspection algorithm and outputting the result as vision inspection result data;
A result delivery step of delivering, by the machine vision device, the vision inspection result data to the central management server (S1430);
Thereafter, the document management step (S1440) of the central management server to automatically create a report-type document from the specific region extracted from the video image of the inspection object and the vision inspection result data;
An edge analysis step (S1450) in which the central management server analyzes edges by processing video images of production facilities;
A monitoring step (S1460) of outputting an image image of an inspection object on a process line, the vision inspection result data, and an image image of production facilities through a screen of a control center in real time;
A prediction step (S1470) of analyzing a report generated during a predetermined period of time to identify a cause of failure or quality deterioration and output facility status prediction data; And
Data linking step (S1480) detects the possibility of a defect or deformation of the production equipment according to the edge analysis results, and associated with the equipment state prediction data
Including,
Test step to test the performance of the machine vision device, automatic document creation performance and image image processing performance of the production facilities
More,
The document creation step (S1440),
A specific region is extracted from the video image of the inspection object by using a position of a predetermined reference point, rotation, filtering, shift, and zoom editing are performed on the detected image, and the vision is performed. Process the inspection result data, and create a document in the report format in real time from the edited image and the processed data,
The preset reference point is,
Characterized in that the portion is printed or displayed on the surface of the inspection object,
The edge analysis step (S1450),
In order to detect edges of the image images of the plurality of production facilities, the pixel value 901 of the input image 900 in the spatial region and the pixel value 901 of the input image 900 are correspondingly multiplied 903 to the output pixel value 904. It uses the image processing line technique to perform a process for calculating the operation for all the input pixel values,
The size of the edge is represented by the following [Equation 5],
[Equation 5]

(Where G (x, y) is the size of the edge, G _x is the row detection mask, and G _y is the column detection mask),
The direction of the edge is characterized in that represented by the following [Equation 6],
[Equation 6]

(Where θ is the direction of the edge, G _x is the row detection mask, G _y is the column detection mask),
The test step,
Vision image detection, characterized in that to generate a virtual test image by applying a different degree of deformation to a random image sample received from a user, and to simulate the operation of the machine vision device using the generated virtual test image Process control and monitoring method using.
delete delete The method of claim 6,
After the data linking step (S1480),
Alarm step (S1490) for outputting warnings and alarms for a facility that requires action or inspection based on the equipment state prediction data and the edge analysis results (S1490)
Process management and monitoring method using a vision image detection, characterized in that it further comprises.
delete

Images