KR102022494B1 - System for automatic generating of documents using vision image detection and a method thereof - Google Patents

Abstract

The present invention relates to a system for automatically generating a document using vision image detection, and to a method thereof. More specifically, provided are the system for automatically generating a document using vision image detection, which receives and stores an image obtained through a plurality of machine vision devices and vision test result data, detects only a necessary area from the obtained image and processes the vision test result data to generate a related document in real time, and analyzes the test result data and predicts a facility requiring the inspection or confirmation in accordance with the analysis result to notify the prediction, thereby replacing repetitive and error-prone handwriting works, and quickly predicting a facility requiring the inspection or conformation to improve the productivity, and the method thereof.

Description

Document automatic creation system using vision image detection and its method {SYSTEM FOR AUTOMATIC GENERATING OF DOCUMENTS USING VISION IMAGE DETECTION AND A METHOD THEREOF}

The present invention relates to an automatic document generation system and method using vision image detection, and more particularly, image and vision inspection result data obtained through a plurality of machine vision devices are received and stored, and required from the acquired image Only the area is detected and the vision inspection result data is processed to create related documents in real time, and the inspection result data is analyzed to predict and notify the equipment that needs to be inspected or checked according to the analysis result, thereby repeating and The present invention relates to an automatic document creation system and method using vision image detection, which can improve the productivity by replacing the manual work of the probable mistake and rapidly predicting the equipment requiring inspection or confirmation.

In recent years, industrial automation has been accelerated by combining elements such as machine vision, robots, sensors, PLCs, computers, etc. across industries such as automobiles, electronics, displays, semiconductors, food and beverage, and sports. have.

As industrial automation is accelerating across industries, a variety of technologies are being introduced to automate the overhaul process that is performed before high volume products are shipped.

Vision inspection (machine vision inspection), one of such precision inspections, is performed by an image taken by a camera instead of a conventional method, which is judged by the human eye, for example, a labeling state of a product, a printing state of a wrapping paper, or a printing state of a barcode. To check the back.

Specifically, the vision inspection is performed by photographing the surface of the inspection object to obtain a surface image, and comparing the obtained surface image with a reference image to inspect the inspection object for defects.

The apparatus for vision inspection is installed in various places on the process line, and performs vision inspection on a product (hereinafter, inspection object) moving along the process line.

In other words, manufacturers form a large-scale production line, and each production line installs a number of vision equipment such as IP cameras to perform defect rate or quality inspection for each process.

On the other hand, in the field of modern manufacturing, image information by vision technology and automatic detection is more reliable and secure than any data. However, there is a lot of inconvenience, such as recording and pasting by hand, or re-recording with a digital camera to create a document because it can not process / edit the important information detected by the vision inspection device, and acts as a production deterrent. This can lead to workers not being able to focus on production and possibly to report something wrong by mistake.

In addition, since the defects of each process are manually linked to each other by document method, the material cost and labor cost increase, and it is difficult to extract the process data. Therefore, process data to be reported when requesting process data from external company or overseas market is manually recorded. You must write it.

In Korean Patent Publication [10-2009-0001710], it is possible to grasp the inventory history and work status of each detailed work line in real time and to understand the re-delivery of subcontractors to maximize manufacturing process efficiency and improve quality control. A manufacturing process management system is disclosed that enables the use of the same.

Korean Laid Open Patent [10-2009-0073643] discloses a production site information providing system and method capable of automatically aggregating production site information of a production facility factory line and providing it to an enterprise resource management system.

On the other hand, Korean Patent Registration [10-1570516] discloses an industrial vision integrated process management system.

On the other hand, the Korean Patent Registration [10-1920372] discloses a visual foresight system of equipment failure and poor quality of the automation line in the advanced product manufacturing plant optimized for small and medium-sized enterprises.

Korean public patent [10-2009-0001710] (published date: 2009. 01. 09) Korean public patent [10-2009-0073643] (published date: 2009. 07. 03) Korea Patent Registration [10-1570516] (Registration Date: November 13, 2015) Korea Patent Registration [10-1920372] (Registration Date: Nov. 14, 2018)

Accordingly, the present invention has been made to solve the above-described problems, an object of the present invention is to receive and store the image and vision inspection result data obtained through a plurality of machine vision devices, and is required from the obtained image Only the area is detected and the vision inspection result data is processed to create related documents in real time, and the inspection result data is analyzed to predict and notify the equipment that needs to be inspected or checked according to the analysis result, thereby repeating and The present invention provides an automatic document creation system and method using vision image detection that can improve the productivity by replacing manual mistakes that are likely to be mistaken, and quickly predicting a facility requiring inspection or confirmation.

The objects of the embodiments of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description. .

In the automatic document creation system using vision image detection according to an embodiment of the present invention for achieving the above object, a plurality of machine vision apparatus for performing a vision inspection of the inspection object to be processed in each work line ( 110, 120, 130); A document automatic creation server (200) for receiving and storing the video image and the vision inspection result data from each machine vision device, and automatically creating a document from the video image and the vision inspection result data; And a terminal (400) for reading the created document, wherein the document automatic creation server (200) comprises: a transceiver (201) for receiving the image image and the vision inspection result data from the machine vision apparatus; An image detector 203 for extracting a specific region from the video image; An image editing unit (205) for editing the detected image; A data processor 206 for processing the vision inspection result data received from the machine vision apparatus; A document creating unit 207 for creating a report form document in real time from the edited image and the processed data; A storage unit 204 for storing the video image and the vision inspection result data received from the plurality of machine vision apparatuses through the transceiver, and storing a document created by the document creating unit; And a controller 202 for controlling each component including the transceiver, the image detector, the image editor, the data processor, and the document generator.

Each machine vision device (110, 120, 130), the camera (301) for photographing the inspection object; An illumination unit 306 for supplying light for photographing the surface of the inspection object; An image preprocessor 302 for preprocessing the image received from the camera; A vision inspection control unit 303 for generating a surface image of the inspection object while controlling the operation of the camera and the lighting unit; Determination unit 304 for determining the good and defective goods through the acquired inspection object image in accordance with a preset vision inspection algorithm; And a communication unit 305 for transmitting the preprocessed image and the vision inspection result determined by the determination unit to the document automatic generation server.

The storage unit 204 may include: an original image storage unit 204a for storing an original image image received from each machine vision apparatus; A detection image storage unit 204b for storing the image detected by the image detection unit; And a document storage unit 205c for storing the document created by the document creating unit.

The document automatic creation server analyzes the vision inspection result data during a predetermined period, and performs process status determination, failure cause analysis, and quality deviation analysis through linkage between processes to identify the influence factors of the defect or quality deviation, and And a data analyzer 208 for outputting state prediction data.

The document created by the document creating unit includes a product name 601, a start time 602, an end time 603, an inspection count 604, a good product 605, a defective 606, a defective rate 607, an edited image ( 608 and a link 609 of the entire image.

In addition, the automatic document creation method using the vision image detection according to an embodiment of the present invention, the photographing step of photographing the inspection object moving along the process line by the machine vision apparatus at a specific position on the process line (S710); An image transfer step of transmitting, by the machine vision apparatus, an image of the inspection object photographed to the document automatic generation server (S720); A vision inspection step (S730) of determining, by the machine vision apparatus, a defective or good product according to a preset vision inspection algorithm of at least one director and outputting it as vision inspection result data; A result delivery step of transmitting, by the machine vision device, the vision inspection result data to the document automatic creation server (S740); An image detection step (S750) of the automatic document generation server detecting a specific area from the video image of the inspection object; And a document creation step (S770) of automatically creating a report form document from the detected image and the vision inspection result data.

After the extracting step (S750), if it is necessary to edit the extracted specific region image further comprises an editing step (S760) for editing the extracted specific region image.

After the document creation step (S770), the vision inspection result data for a predetermined period of time is analyzed to determine process factors, defect cause analysis, and quality deviation analysis through linkage between processes to determine the influence factors of defects or quality deviations. And, it characterized in that it further comprises a prediction step (S780) for outputting the equipment state prediction data.

After the prediction step (S780), based on the prediction data, characterized in that it further comprises an alarm step (S790) for outputting warnings and alarms for the equipment requiring action or inspection.

The automatic document creation method using the vision image detection may further include a test step for testing the performance of the vision inspection apparatus and the automatic document creation performance.

According to an automatic document generation system using a vision image detection and the method according to an embodiment of the present invention, the image and vision inspection result data obtained through a plurality of machine vision apparatus is received and stored, and is required from the acquired image Only the area is detected and the vision inspection result data is processed to create related documents in real time, and the inspection result data is analyzed to predict and notify the equipment that needs to be inspected or checked according to the analysis result, thereby repeating and Productivity can be improved by replacing manual mistakes that are likely to be mistaken and quickly predicting equipment that needs inspection or verification.

In addition, according to the automatic document creation system and method using a vision image detection according to an embodiment of the present invention, producers or line managers can focus on production without the need to create a separate report, which may occur when making a report Mistakes can be prevented beforehand.

In addition, according to the automatic document creation system and method using a vision image detection according to an embodiment of the present invention, there is an effect that can be monitored in real time and real-time remotely through the image and inspection data stored in a database (storage device) .

In addition, according to the automatic document creation system and method using the vision image detection according to an embodiment of the present invention, by collecting and monitoring the data transmitted from the machine vision device to find a facility that affects the defect or quality, quality Analyzing the deterrents has the effect of quality history management / tracking, cause analysis for major facilities that have been found to be factors of quality, generation of predictive data for future facilities, and improvement of quality and yield.

1 is a block diagram of an automatic document creation system using vision image detection according to an embodiment of the present invention.
2 is a detailed configuration diagram of an embodiment of an automatic document generation server of FIG. 1.
Figure 3 is a detailed configuration diagram of one embodiment of the machine vision device of FIG.
4A and 4B illustrate an embodiment of a method for editing a vision image according to the present invention.
5 is a block diagram of an image test apparatus for image testing further included in an automatic document generation system using vision image detection according to an embodiment of the present invention.
6A and 6B are diagrams illustrating an embodiment of a video image and a document stored according to a method for automatically creating a document using vision image detection according to the present invention.
6C is an exemplary explanatory diagram of a document created according to an automatic document creation method using vision image detection according to the present invention;
7 is a flowchart illustrating an embodiment of a method for automatically creating a document using vision image detection according to the present invention.
8 is a view for explaining a facility image image processing method for facility inspection in the automatic document creation system using vision image detection according to the present invention.
FIG. 9A is a mask according to an embodiment of the present invention used in processing a facility image for facility inspection in an automatic document generation system using vision image detection according to the present invention. FIG.
9B is an explanatory diagram of an image processed using the mask of FIG. 9A;
Figure 10a is a mask according to another embodiment used in the facility image image processing for facility inspection in the automatic document creation system using vision image detection according to the present invention.
10B is an explanatory diagram of an image processed using the mask of FIG. 10A;
Figure 11a is a mask according to another embodiment used in the facility image image processing for facility inspection in the automatic document creation system using vision image detection according to the present invention.
11B is an explanatory diagram of an image processed using the mask of FIG. 11A.
12A and 12B illustrate an exemplary embodiment for explaining before and after facility image image processing for facility inspection in a document automatic generation system using vision image detection according to the present invention;

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be.

On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "having" are intended to indicate that there is a feature, number, process, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present disclosure does not exclude the possibility of the presence or the addition of numbers, processes, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings will be described in more detail the present invention. Prior to this, terms or words used in the present specification and claims should not be construed as being limited to the ordinary or dictionary meanings, and the inventors should properly explain the concept of terms in order to best explain their own inventions. Based on the principle that it can be defined, it should be interpreted as meaning and concept corresponding to the technical idea of the present invention. In addition, unless there is another definition in the technical terms and scientific terms used, it has the meaning commonly understood by those of ordinary skill in the art to which this invention belongs, and the gist of the present invention in the following description and the accompanying drawings. Descriptions of well-known functions and configurations that may be unnecessarily blurred are omitted. The drawings introduced below are provided by way of example so that the spirit of the invention to those skilled in the art can fully convey. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. Also, like reference numerals denote like elements throughout the specification. It should be noted that the same elements in the figures are represented by the same numerals wherever possible.

In an embodiment of the present invention, a food is made as an object to be inspected and manufactured by an automated process, and a case in which a printing state of food is printed on a machine vision device will be described as an example.

1 is a block diagram of an automatic document generation system using vision image detection according to an embodiment of the present invention.

As shown in FIG. 1, a document automatic creation system using vision image detection according to an embodiment of the present invention includes a plurality of machine vision devices 110, 120, and 130 document automatic creation server 200, and a terminal 400. ).

Each of the machine vision devices 110, 120, and 130 is a device that performs machine vision inspection.

At this time, the machine vision inspection inspects the property of the object using one or more algorithms of pattern matching, point / line / face fitting, edge, color, and location. It is.

An application (ie, vision software) may be installed on the machine vision devices 110, 120, and 130 for machine vision inspection, and the application may acquire a target image of the inspection object and acquire the target image of the machine vision device 110 according to an algorithm. , 120 and 130 may be controlled.

In addition, the machine vision apparatuses 110, 120, and 130 may include a camera (not shown) for capturing an object, or may perform a machine vision inspection by communicating with a camera (not shown) located outside.

The machine vision apparatuses 110, 120, and 130 are interspersed in a plurality of process lines, and a plurality of machine vision apparatuses 110, 120, and 130 are installed, and transmit image information of real-time process states for each process state.

The machine vision apparatuses 110, 120, and 130 may have different APIs (application programming interfaces) and communication protocols supported by manufacturers, such as SDK, RTSP, HTTP CGI, and TCP / IP.

The document automatic creation server 200 receives and stores an image image and a vision inspection result data from each of the machine vision devices 110, 120, and 130, and automatically creates a document from the image image and the vision inspection result data. do.

The terminal 400 allows to view the created document.

In this case, the terminal 400 may be a person in charge PC or board room PC for each process line, the monitoring information, the degree of participation of each process status or statistics and report creation or reading level is set according to the preset management grade.

For example, when the terminal 400 is the person in charge of the PC, process state image information is transmitted to monitor the process state of the corresponding process line, and statistics on the process state after checking the defective rate by analyzing the process state image information And reports.

On the other hand, when the terminal 400 is a boardroom PC, based on the statistics and reports received from the person in charge PC and the process state image information of each process line to monitor the current production rate and the defective rate for each process, the specific process through You can also send a work order to the person in charge.

On the other hand, the terminal 400 is at least one multimedia terminal including a mobile terminal such as a smart device such as a PC, a smart phone, a smart pad and a tablet PC, a mobile phone, a PDA, a notebook, and the like through a communication network vision image It is connected with automatic document creation system using detection.

Here, the communication network may be configured without regard to communication modes such as wired and wireless, and may include a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN). It may be configured with a variety of communication networks. Preferably, the communication network referred to in the present invention may be a known World Wide Web (WWW).

In addition, the automatic document generation system using vision image detection according to an embodiment of the present invention may further include a vision inspection test apparatus 300.

The vision inspection test apparatus 300 may determine the performance of the machine vision apparatus and determine whether the automatic document creation is properly performed. The detailed configuration of the vision inspection test apparatus 300 will be further described with reference to FIG. 5.

FIG. 2 is a detailed block diagram of an embodiment of an automatic document generation server of FIG. 1.

As illustrated in FIG. 2, the document automatic creation server 200 may include a transceiver 201, an image detector 203, an image editor 205, a data processor 206, a document generator 207, and a storage unit ( 204, and a control unit 202.

The transceiver 201 receives the video image and the vision inspection result data from the machine vision apparatuses 110, 120, and 130.

The image detector 203 extracts a specific area from the video image.

As a method of extracting a specific region, a position of a predetermined reference point may be used in the photographed image of the inspection object. The predetermined reference point may be a portion printed or displayed on the surface of the inspection object.

The image editing unit 205 edits the detected image.

The data processor 206 processes the vision inspection result data received from the machine vision apparatus.

The document creating unit 207 creates a report form document in real time from the edited image and the processed data.

The storage unit 204 stores the video image and the vision inspection result data transmitted from the plurality of machine vision devices through the transceiver unit, and stores the document created by the document preparation unit.

The storage unit 204 may include an original image storage unit 204a storing an original image image transmitted from each machine vision apparatus, a detection image storage unit 204b storing an image detected by the image detection unit, and And a document storage section 205c for storing a document created by the document creating section.

The controller 202 controls each component including the transceiver 201, the image detector 203, the image editor 205, the data processor 206, and the document generator 207. .

The document automatic creation server 200 analyzes the vision inspection result data during the preset period, and performs process status determination, failure cause analysis, and quality deviation analysis through linkage between processes to identify the influence factors of the defect or quality deviation. The apparatus may further include a data analyzer 208 for outputting the facility condition prediction data.

3 is a detailed block diagram of an embodiment of the machine vision apparatus of FIG. 1.

As shown in FIG. 3, each machine vision apparatus includes a camera 301, an image preprocessor 302, a vision inspection control unit 303, a determination unit 304, a communication unit 305, and an illumination unit 306. .

The camera 301 photographs an inspection object and provides an image photographed to the image preprocessor 302 which will be described later.

The illumination unit 306 emits illumination light for photographing the surface image, and emits light toward the inspection object while emitting illumination in front of the camera 301. The lighting unit 306 may be configured as a substrate on which a plurality of LED elements are mounted, and may be preferably disposed behind the camera 301.

The image preprocessing unit 302 preprocesses an image of an inspection object and stores the image in a storage unit (not shown), or the preprocessed image is stored in a separate storage device (eg, automatic document generation through the communication unit 305). Storage unit 204 of the server 200).

The vision inspection control unit 303 is a component that generates the surface image of the inspection object while controlling the operations of the camera 301 and the lighting unit 306 described above.

The vision inspection control unit 303 controls the overall operation of the machine vision apparatus, and may include a processor such as a CPU. The vision inspection controller 303 may control other components included in the machine vision apparatus to perform an operation corresponding to a user input received through an input / output unit (not shown).

For example, the vision inspection controller 303 may execute a program stored in the memory, read a file stored in the memory, or store a new file in the memory.

Here, the vision inspection control unit 303 sets the photographing conditions of the camera 301 (for example, the exposure value or the sensitivity of the camera 301, the aperture setting, the shutter speed, etc.) while generating the surface image of the inspection object. In addition, by generating a plurality of surface images, it is possible to generate a high quality image.

The determination unit 304 determines a good or defective product through an image of the inspection object obtained according to a predetermined vision inspection algorithm, and stores the vision inspection result thereof in a storage unit (not shown), or the vision inspection. The result is stored in a separate storage device (eg, the storage unit 204 of the document automatic creation server 200) through the communication unit 305.

That is, the image acquired through the camera 301 is interpreted through vision software. Vision software can match patterns of objects from the extracted images, fit points, lines or faces, distinguish colors, gauge objects, or provide location information for robot guides. In addition, one-dimensional and two-dimensional barcodes displayed on the object may be read or optical character reading (OCR) may be performed. This process allows objects to be inspected, measured or read, enabling industrial automation.

4A and 4B are diagrams illustrating an example of a method of editing a vision image in the image editing unit 205 of the document automatic generation server 200 according to the present invention.

4A and 4B show images 410 and 420 received from the machine vision devices 110, 120 and 130.

In the first image 410, the inspection object 411 may be correctly placed and photographed to extract the distribution period region 412 in a quadrangular shape.

On the other hand, in the second image 420, when the inspection object 421 is photographed without being properly placed and the distribution period region 422 is extracted, an operation of rotating is required.

On the other hand, when the image sharpness of the region to be extracted is inferior, the image editing unit 205 may perform an operation of increasing the sharpness through filtering.

In addition, the image editing unit 205 may edit not only the rotation but also the shift and the zoom.

5 is a detailed configuration diagram of a vision inspection test apparatus for performance testing, which is further included in an automatic document generation system using vision image detection according to an embodiment of the present invention.

As shown in FIG. 5, the vision inspection test apparatus 300 includes a test image generator 501 and a simulation unit 502.

The test image generator 501 generates a virtual test image for simulating the operation of the machine vision apparatus 110, 120, 130, and the test image generator 501 receives input from a user or a random image sample. Create a test image with different degrees of variation.

The degree of deformation can be divided into several stages, and the factors of deformation include brightness, noise, contrast, and shading, depending on vibration, camera positional deviation, poor focus, foreign objects and damage. ), Shift, rotation, lens flare, zoom, balance, blur, dust, scratch, and the like.

The simulation unit 502 simulates the operations of the machine vision devices 110, 120, and 130 using the virtual test image generated by the test image generation unit 501.

Each of the machine vision apparatuses 110, 120, and 130, the test image generator 501, and the simulation unit 502 may be included in or included in each other. That is, for example, the test image generator 501 including the simulation unit 502 may be included in the machine vision devices 110, 120, and 130, or may generate a test image including the simulation unit 502. The unit 501 may include the machine vision devices 110, 120, and 130, or the simulation unit 502 including the machine vision devices 110, 120, and 130 may include the test image generator 501. It may be located outside and communicate with the test image generator 501.

In addition, each of the machine vision devices 110, 120, and 130, the test image generator 501, and the simulation unit 502 may be located outside of each other and communicate with each other through a network.

As illustrated in FIG. 5, the vision inspection test apparatus 300 including the test image generator 501 and the simulation unit 502 may be provided in a separate PC or may be driven under the operation of vision software.

The vision inspection test apparatus 300 may perform a test simulation operation of the machine vision apparatus and a number of test image generations, a test image deformation degree setting according to a user's setting input value through a vision inspection application (vision software).

6A and 6B are diagrams illustrating an embodiment of a video image and a document stored according to an automatic document generation method using vision image detection according to the present invention.

FIG. 6A illustrates that data received from a machine vision apparatus is organized into folders for each date according to a method for automatically creating a document using vision image detection according to the present invention. In FIG. Shows that there are folders and auto-created documents.

6C is a diagram illustrating an embodiment of a document created according to an automatic document generation method using vision image detection according to the present invention.

As shown in Figure 6c, looking at the document created by the document automatic creation method using the vision image detection according to the present invention, product name 601, start time 602, end time 603, inspection count 604 , The article 605, the defective 606, the defective rate 607, the edited image 608, and the link 609 of the whole image.

That is, in the automatic document creation system according to the present invention, the vision inspection result is received to generate a report type document in real time.

7 is a flowchart illustrating an embodiment of a method for automatically creating a document using vision image detection according to the present invention.

First, the machine vision apparatus 110, 120, 130 photographs an inspection object moving along a process line at a specific position on the process line (S710).

Thereafter, the machine vision apparatuses 110, 120, and 130 transmit the video image of the photographed inspection object to the document automatic creation server 200 (S720).

Thereafter, the machine vision apparatuses 110, 120, and 130 analyze the video image of the photographed inspection object according to a preset vision inspection algorithm, determine a defective or good product, and output it as vision inspection result data (S730).

Thereafter, the machine vision apparatuses 110, 120, and 130 transmit the vision inspection result data to the document automatic creation server 200 (S740).

Here, the video image of the inspection target and the vision inspection result data may be transferred to the document automatic creation server 200 together. The video image of the inspection object and the inspection result data are associated and stored in the storage unit 204 of the document automatic creation server 200.

Thereafter, the document automatic generation server 200 extracts a specific region from the video image of the inspection object (S750).

Subsequently, when editing of the extracted specific region image is necessary, the editing is performed (S760).

Thereafter, a document in a report format is automatically created from the edited image and the vision inspection result data (S770).

Then, by analyzing the documents created during the predetermined period of time to identify the factors of poor or poor quality and outputs the equipment condition prediction data (S780).

Then, the warning and alarm for the equipment that needs to take action or check based on the predicted data (S790).

Although not shown in the drawings, the automatic document creation method using vision image detection according to the present invention may further include a test step of determining the performance of the machine vision apparatus and thereby verifying whether automatic document creation is performed without a problem. Can be. In the test step, by generating a test image, and input it to the machine vision device to simulate, it is possible to know the performance of the machine vision device and the automatic document creation performance.

On the other hand, the edge (Edge) refers to the portion where the brightness is sharply changed in the image, it means a boundary line or contour. By detecting the edge, information about the position, shape, size, etc. of the object can be known. Edge detection is the process of finding edge pixels, and edge enhancement is to increase the contrast between the edge and the background to make the edge more visible. Edge tracing is the process of following an edge.

An edge means a place where a large difference in pixel values occurs, such as a discontinuous region of pixel values, and thus, the detection of the edge uses a concept of differential operation for measuring spatial variation. That is, since the edge is the rate of change of contrast or brightness based on the contrast of the image, that is, the slope. The slope in the image is called a gradient, and you can get the edge by getting the gradient.

Gradient is an extension of the first derivative to two dimensions (x, y) and is a measure of the measure of variation in the function.

The edge detection methods include a detection method using a first derivative and a detection method using a second derivative. The detection method using the second derivative is a method of improving the performance of edge detection by adding the derivative once more to the result obtained as the first derivative.

8 is a view for explaining a facility image image processing method for facility inspection in the automatic document creation system using vision image detection according to the present invention.

As illustrated in FIG. 8, the facility inspection unit (not shown) included in the automatic document generation system using vision image detection according to the present invention is an image processing line for processing image images for edge detection of production facilities. Use the Convolution technique.

It is called spatial area based processing or mask based processing that performs operations based on a prescribed area called a mask, not pixel based processing that performs various operations on each pixel value.

Mask-based processing calculates the output pixel value 804 by correspondingly multiplying the element values in the mask 802 by the pixel values 801 of the input image 800 in the spatial domain. Performing this process while moving for all input pixel values is called a line (convolution). In this case, a mask multiplied by the input image is called a kernel, a window, a filter, or the like.

That is, the new output pixel value 804 is a value obtained by multiplying the pixel values 801 of the input image by the weights of the corresponding line masks 802 and adding them together (803).

 Examples of first order derivatives include the Robert operator, the Sobel operator, and the Prewitt operator. This will be described with reference to FIGS. 9A to 11B.

FIG. 9A is a mask according to an exemplary embodiment used in processing a facility image for facility inspection in a document automatic generation system using vision image detection according to the present invention.

9A shows the row detection mask G _x and the column detection mask G _y of the Robert operator.

The Robert operator is smaller in size than other masks and can be used effectively, but has a disadvantage of being sensitive to noise because it is difficult to use the extruded values appropriately for the average.

9B is an explanatory diagram of an image processed using the mask of FIG. 9A.

Referring to FIG. 9B, there is an original image on the upper left, a horizontal image using a row detection mask on the upper right, a vertical image using a column detection mask on the lower left, and a row detection mask and a column detection mask on the lower right. The image using both is shown.

FIG. 10A is a mask according to another exemplary embodiment used in processing a facility image for facility inspection in a document automatic generation system using vision image detection according to the present invention.

10A illustrates a row detection mask G _x and a column detection mask G _y of the Sobel operator.

 The Sobel operator has the advantage of further emphasizing the center value of the original image, and is efficient at averaging protrusion values, but sensitive to diagonal edges.

10B is an explanatory diagram of an image processed using the mask of FIG. 10A.

Referring to FIG. 10B, there is an original image on the upper left, a horizontal image using a row detection mask on the upper right, a vertical image using a column detection mask on the lower left, and a row detection mask and a column detection mask on the lower right. The image using both is shown.

FIG. 11A is a mask according to another exemplary embodiment used in processing a facility image for facility inspection in a document automatic generation system using vision image detection according to the present invention.

11A shows a row detection mask G _x and a column detection mask G _y of a Prewitt operator.

The Prewitt operator has the same advantages as the Sobel filter, with slightly faster response times and relatively efficient averaging of the protruding values, but sensitive to horizontal and vertical edges, It is less pronounced.

11B is an explanatory diagram of an image processed using the mask of FIG. 11A.

Referring to FIG. 11B, there is an original image on the upper left, a horizontal image using a row detection mask on the upper right, a vertical image using a column detection mask on the lower left, and a row detection mask and a column detection mask on the lower right. The image using both is shown.

On the other hand, the size of the edge can be expressed as shown in [Equation 1].

[Equation 1]

Here, G (x, y) is an edge size, G _x is a row detection mask, and G _Y is a column detection mask.

On the other hand, the direction of the edge can be expressed as shown in [Equation 2].

[Equation 2]

Is the direction of the edge, G _x is the row detection mask, and G _Y is the column detection mask.

12A and 12B illustrate an exemplary embodiment for explaining before and after facility image image processing for facility inspection in a document automatic generation system using vision image detection according to the present invention.

12A is an image before processing an image image of a production facility in a facility inspection unit (not shown) provided in the automatic document generation system using vision image detection according to the present invention, and FIG. 12B is a facility inspection using a Sobel mask. This is the image after the department processes the video image of the production equipment.

Referring to FIGS. 12A and 12B, by processing video images of a production facility, edge detection is possible, and changes in damage, defects, and deformation of the production facility may be detected.

Therefore, in addition to the vision inspection results on the product (inspection object) in the process, as well as connected to each other through the detection of changes in the appearance of the production equipment, the failure prediction and defect prevention of the equipment can be made quickly, productivity can be improved.

Although the automatic document creation method using the vision image detection according to an embodiment of the present invention has been described above, the computer-readable recording medium and the vision image detection in which a program for implementing the automatic document creation method using the vision image detection are stored. The program stored in the computer-readable recording medium for implementing the automatic document creation method may also be implemented.

That is, it will be readily understood by those skilled in the art that the above-described method for automatically creating a document using vision image detection may be provided included in a recording medium that can be read through a computer by program of instructions for implementing the same. will be. In other words, it may be embodied in the form of program instructions that can be executed by various computer means, and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in computer software. Examples of such computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and floptical disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, USB memory, and the like. The computer-readable recording medium may be a transmission medium such as an optical or metal wire, a waveguide, or the like including a carrier wave for transmitting a signal specifying a program command, a data structure, or the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

The present invention is not limited to the above-described embodiments, and the scope of application is not limited, and various modifications can be made without departing from the gist of the present invention as claimed in the claims.

110, 120, 130: machine vision device 200: document auto-creation server
300: vision inspection test device 400: terminal
201: transceiver unit 202: controller
203: Image detector 204: Storage unit
205: image editing unit 206: data processing unit
207: document creation unit 208: data analysis unit
301: camera 302: image preprocessor
303: vision inspection control unit 304: determination unit
305: communication unit 306: lighting unit
501: Test image generation unit 502: Simulation unit

Claims (10)

In the document automatic creation system using vision image detection,
A plurality of machine vision devices (110, 120, 130) for performing vision inspection of the inspection object processed in each work line;
A document automatic creation server (200) for receiving and storing the video image and the vision inspection result data from each machine vision device, and automatically creating a document from the video image and the vision inspection result data;
A terminal 400 for reading the created document; And
Vision inspection test apparatus 300 for testing the performance of the plurality of machine vision devices, the automatic creation of the document
Including,
The document automatic creation server 200,
Transmitting and receiving unit 201 for receiving the video image and the vision inspection result data from the machine vision device;
An image detector 203 for extracting a specific region from the video image;
An image editing unit (205) for editing the detected image;
A data processor 206 for processing the vision inspection result data received from the machine vision apparatus;
A document creating unit 207 for creating a report form document in real time from the edited image and the processed data;
A storage unit 204 for storing the video image and the vision inspection result data received from the plurality of machine vision apparatuses through the transceiver, and storing a document created by the document creating unit;
A controller 202 for controlling each component including the transceiver, the image detector, the image editor, the data processor, and the document generator; And
To analyze the vision inspection result data during the preset period, determine process factors, defect cause analysis, and quality deviation analysis through inter-process linkages to identify the influence factors of defects or quality deviations, and to output facility status prediction data. Data Analysis Unit (208)
Including,
The document created by the document creation unit,
Product Name (601), Start Time (602), End Time (603), Inspection Count (604), Good Quality (605), Defect (606), Defective Rate (607), Edited Image (608), and Link to Full Image (609) )
Characterized in that it comprises a,
Facility inspection unit for processing video images for edge detection on production facilities
More,
The image detector 203,
Using a position of a predetermined reference point in the photographed video image of the inspection object, wherein the predetermined reference point is a portion that is printed or displayed on the surface of the inspection object,
The image editing unit 205,
Rotation, filtering, shift, and zoom editing are performed on the detected image.
Regarding the video image of the production facilities,
The process of calculating the output pixel value 804 by correspondingly multiplying 803 the element values in the mask 802 and the pixel values 801 of the input image 800 in the spatial area for edge detection to calculate all the input pixel values. Characterized by using an image processing line technique that is performed while moving about,
The size of the edge is represented by the following [Equation 3],
[Equation 3]

(Where G (x, y) is the size of the edge, G _x is the row detection mask, and G _y is the column detection mask),
The direction of the edge is characterized in that represented by the following [Equation 4],
[Equation 4]

(Where θ is the direction of the edge, G _x is the row detection mask, G _y is the column detection mask),
The vision inspection test device 300,
A test image generator 501 for generating a virtual test image by applying a different degree of deformation to a random image sample received from a user; And
A simulation unit 502 for simulating the operation of the machine vision device by using the virtual test image generated by the test image generator.
Automatic document generation system using a vision image detection, comprising a.
The method of claim 1,
Each machine vision device (110, 120, 130),
A camera 301 for photographing the inspection object;
An illumination unit 306 for supplying light for photographing the surface of the inspection object;
An image preprocessor 302 for preprocessing the image received from the camera;
A vision inspection control unit 303 for generating a surface image of the inspection object while controlling the operation of the camera and the lighting unit;
Determination unit 304 for determining the good and defective goods through the acquired inspection object image in accordance with a preset vision inspection algorithm; And
The communication unit 305 for transmitting the pre-processed image and the vision inspection result determined by the determination unit to the document automatic creation server
Automatic document generation system using a vision image detection, comprising a.
The method of claim 1,
The storage unit 204,
An original image storage unit 204a for storing an original image image received from each machine vision device;
A detection image storage unit 204b for storing the image detected by the image detection unit; And
A document storage unit 205c storing a document created by the document creating unit
Automatic document generation system using a vision image detection, comprising a.
delete delete In the document automatic creation method using vision image detection in the automatic document creation system using the vision image detection of claim 1,
A photographing step of photographing, by a machine vision apparatus, an inspection object moving along a process line at a specific position on the process line (S710);
An image transfer step of transmitting, by the machine vision apparatus, an image of the inspection object photographed to the document automatic generation server (S720);
A vision inspection step (S730) of determining, by the machine vision apparatus, at least one defective or good product according to a preset vision inspection algorithm and outputting the vision inspection result data as vision inspection result data;
A result delivery step of transmitting, by the machine vision device, the vision inspection result data to the document automatic creation server (S740);
An image detection step (S750) of the automatic document generation server detecting a specific area from the video image of the inspection object;
An editing step of editing the extracted specific region image when editing of the detected specific region image is necessary (S760);
A document creation step (S770) of automatically creating a report type document from the detected image and the vision inspection result data; And
Analyzing vision inspection result data for a preset period of time, conducting process status determination, failure cause analysis, and quality deviation analysis through linkage between processes to identify the influence factors of defects or quality deviations, and output facility status prediction data Step (S780)
Including,
A testing step for testing the performance of the machine vision device and the automatic document writing performance; And
A facility inspection step of processing an image image for edge detection for production facilities,
Document creation step (S770),
A specific region is extracted from the video image of the inspection object by using a position of a predetermined reference point, rotation, filtering, shift, and zoom editing are performed on the detected image, and the vision is performed. Process the inspection result data, and create a document in the report format in real time from the edited image and the processed data,
The document created in the document creation step,
Product Name (601), Start Time (602), End Time (603), Inspection Count (604), Good Quality (605), Defect (606), Defective Rate (607), Edited Image (608), and Link to Full Image (609) )
Characterized in that it comprises a,
The preset reference point is,
Characterized in that the portion is printed or displayed on the surface of the inspection object,
The facility inspection step,
The process of calculating the output pixel value 804 by correspondingly multiplying 803 the element values in the mask 802 and the pixel values 801 of the input image 800 in the spatial area for edge detection to calculate all the input pixel values. Characterized by using an image processing line technique that is performed while moving about,
The size of the edge is represented by the following [Equation 5],
[Equation 5]

(Where G (x, y) is the size of the edge, G _x is the row detection mask, and G _y is the column detection mask),
The direction of the edge is characterized in that represented by the following [Equation 6],
[Equation 6]

(Where θ is the direction of the edge, G _x is the row detection mask, G _y is the column detection mask),
The test step,
Vision image detection, characterized in that to generate a virtual test image by applying a different degree of deformation to a random image sample received from a user, and to simulate the operation of the machine vision device using the generated virtual test image Automatic document creation method using.
delete delete The method of claim 6,
After the prediction step (S780),
Alarm step (S790) for outputting warnings and alarms for the facility that needs to take action or check based on the prediction data
Automatic document creation method using a vision image detection, characterized in that it further comprises. delete

Images