KR20100088410A - System for intelligent quality judgment based on htm - Google Patents

Abstract

PURPOSE: An intelligent quality estimation system based on a HTM(Hierarchical Temporal Memory) is provided to determine whether a product is defect based on the quality of a corresponding product by automatically determining the type of the product. CONSTITUTION: A product information is inputted in an item reference information input stage(S100). Products production facility information and sensor information are inputted in a facility information input stage(S110). A product test condition is set in a test condition setting stage(S120). A criteria for determining whether a product has a defect is set in a quality standard setting stage(S130). A control method about a fault is set in a facility control method setting stage(S140). A product is inspected through an image reading process, an image editing process, and an image storing process in a product inspection stage(S150).

Description

HTM based intelligent quality judgment system {System for intelligent quality judgment based on HTM}

The present invention relates to an intelligent quality determination system, and more particularly, to an intelligent quality determination system for determining the quality of a product based on HTM.

In the related art, in order to determine the quality of a product formed by a work such as forging, an operator visually judges the product. In other words, by visually recognizing each product by the individual judgment of the operator to determine the defect and good quality.

Therefore, in the case of the worker's judgment, because the product must be inspected one by one, there is a problem that the work efficiency is reduced. In addition, inspection standards may change depending on the worker's situation. That is, there is a problem that the quality judgment is subjective because it can be changed in the standard of defective and good quality according to the condition of the worker on that day.

Recently, a method of recognizing the shape of a product by artificial intelligence or neural network theory has emerged. However, even in the case of artificial intelligence or neural network theory, all the relevant criteria should be programmed and specified basically, such as setting the product and assigning a variable for it.

In other words, even when judging the shape of a product by conventional artificial intelligence or neural network theory, input neurons of many neural networks are required, so that the learning time of such neural networks is excessively excessive and computational time is required. In addition, since the shape determination is possible only for a specific corresponding product, it is cumbersome to modify the program each time and set the standard of the corresponding product in order to determine the shape of a plurality of products.

An object of the present invention is to solve the problems of the prior art as described above, using a recognition engine based on HTM (HTM) for a number of products coming out of the manufacturing process for each product by using only the image of each It is to provide HTM based intelligent quality judgment system to judge product defects and good quality.

Another object of the present invention is to provide an HTM-based intelligent quality determination system that controls a facility or transmits a defect state to the outside when product defects are continuously made.

HTM based intelligent quality determination system according to the present invention for achieving the above object,

An item reference information input step of inputting information of a product having good quality; A facility information input step of inputting product production facility information for producing a product and sensor information for measuring a state of an ambient environment such as pressure or temperature; An inspection condition setting step of setting a condition in terms of time or quantity in which a product inspection is to be performed; A quality standard setting step of setting each of a plurality of products how to judge defective and good products respectively; A facility control method setting step of setting a control method on how to control the operation of the facility in case of a failure; An image reading process of reading an image by taking an image of a product, an image editing process of editing an image read in the image reading process, and an image storing process of storing an image manipulated by the image editing process. A product inspection step of inspecting the product through; A product recognition step of recognizing the image of the product measured in the product inspection step to recognize the type of the product and the correct product shape; A product quality judging step of judging defects and good products by comparing the image finally recognized in the product recognition step with the reference image of the corresponding product; And when a product defect occurs by the product quality determination step, a subsequent action step of controlling the facility according to the control method set in the facility control method setting step at the same time as notifying the fact to the outside.

According to the intelligent quality determination system of the present invention as described above, the present invention is basically based on HTM technology.

Therefore, the quality can be determined by learning without a separate operation for a number of products, there is an advantage that the work efficiency is improved, the quality function is improved. That is, according to the present invention, even when the type of the product is different, the corresponding type is automatically determined by learning to determine whether the product is defective based on the quality standard of the corresponding product. Thus, according to the present invention, since it is possible to learn and judge closer to human beings, quality judgment work becomes more effective.

Further, in the present invention, an image inputted by crop and contrast is manipulated, and the image is read out step by step at four levels. Accordingly, there is an advantage in that defect determination through an image is easier and a time delay for image recognition is prevented.

In addition, in the present invention, when the failure of the product is made continuously, the equipment is controlled to stop the equipment. In addition, the buzzer rings to notify the outside of the occurrence of product defects, and also notify the remote user of the quality status through SMS. Therefore, mass defects can be prevented in advance, thereby preventing the waste of materials and improving the reliability of the product.

Hereinafter, an HTM based intelligent quality determination system will be described.

The present invention is basically a system for judging quality by a recognition engine based on HTM.

Hierarchical Temporal Memory (HTM) model refers to the mechanism of the neocortex that governs human intelligence. In other words, Hawkins is a computational model of the neocortex proposed by modeling the behavior of neurons and synapses in the human brain.

This Hierarchical Temporal Memory (HTM) model is an artificial neural network based on conventional heuristic search (Artificial intelligence) or simple neurons (Artificial neural) network (ANN)).

In the HTM, Hierarchical Temporal Memory (HTM) is a basic unit of the neocortex consisting of six layers of a node, which is a basic unit of a network, that is, a neocortex column is a unit node. (Consisting of approximately 1,000 neurons and 1,000,000 synapses), they are organized into a hierarchy network, which uses the world's spatial-temporal pattern information ( In particular, temporal information can be stored to make intelligent judgment efficiently.

Therefore, the recognition of the world is more efficient and more robust than the existing AI or ANN.

More specifically, Hierarchical Temporal Memory (HTM), in the sense of hierarchical, is organized into a hierarchical group of nodes, each node having a learning and memory function and performing this. Has an internal algorithm. The lower level node receives a large amount of input from the outside, processes it, and sends the processing result to the next higher level.

Ii) In the sense of Temporal, during training, the HTM application presents the object in a form that changes over time, for example, while training a drawing application, the image is from top to bottom and from left to right. It is represented as if the image is moving over time. And this temporal factor is very important, and the algorithm is expected to expect inputs that change over time.

I) In the sense of memory, HTM applications operate in two stages. That is, it trains memory and uses it for inference. During training, the HTM network learns to recognize patterns in the input it is accepting, where it is trained separately at each level of the hierarchy.

In this way, when the network is fully trained, each level in the hierarchy will systematically have all the objects in its world in memory.

Next, while inferring and recognizing, the HTM network receives the new object and determines that it is most likely one of the objects it already knows.

This hierarchical temporal memory (HTM) is a new computing paradigm that software-models the neocortex of the human brain to enable computers to solve problems that are easy for humans but hard for machines to solve.

With HTM, you can discover patterns in the incoming sensor data, and in some simple applications (image recognition, diagram recognition, wave recognition), the performance excellence has already been demonstrated.

The present invention, by using such HTM, can be utilized in the judgment for quality assurance of the manufacturing plant. That is, the system can determine the quality situation in or between processes using data collected from various sensors (image, temperature, humidity, dust).

As described above, in a recognition engine based on HTM, when a plurality of images are mixed, the plurality of images are classified and judged by similar types. This is as if children were trained and recognized by pictures or in real life, and similar shapes would be recognized as trains and planes, regardless of their size.

1 schematically illustrates a recognition function by an HTM-based recognition engine.

Here, among the plurality of images on the left side, images having a triangular shape are recognized as tents regardless of the height or the width of the left and right sides, and the cylindrical images are cylinders. It is recognized as (Cylinder), and the images of the rectangular shape having a sharp upper surface are recognized as a house.

The present invention can be used to determine the quality of various products using the HTM-based recognition engine, such as changing the program even if the product is changed or a large number of products at the same time or change the program Quality judgment is possible based on learning without changing equipment.

2 is a block diagram showing a schematic configuration of an HTM based intelligent quality determination system according to the present invention.

As shown in the figure, HTM-based intelligent quality determination system according to the present invention, the item reference information input step (S100) for inputting the information of the product having a good quality, and a product production facility for producing a product Equipment information input step (S110) for inputting information and sensor information for measuring environmental conditions such as pressure or temperature, and inspection condition setting step for setting conditions in a timely or quantitative manner in which a product inspection is to be performed (S120), and the quality standard setting step (S130) for setting each of the plurality of products how to determine the criteria of defectiveness and good quality, and a control method of how to control the operation of the equipment in the event of a failure In the facility control method setting step (S140) and the product inspection step (S150) and the product inspection step (S150) to inspect the product by taking a picture of the product The product recognition step (S160) of recognizing the correct product shape by specifying the image of the determined product and the product recognized by comparing the image finally recognized in the product recognition step (S160) and the reference image of the corresponding product If product defects are caused by the quality determination step (S170) and the product quality determination step (S170), the facility is controlled according to the control method set in the facility control method setting step (S140) at the same time as a notification of the fact to the outside. It consists of a subsequent action step (S180) and the like.

The item criterion information input step (S100) is a process of inputting specifications of a good product. That is, the process of inputting the product image of the good product, the image (Image) of such a product in the case of a thin product to enter the front or back of the product, and also to process the image (Image) of both the front and back All is possible.

The facility information input step (S110) is a process of inputting facility and other sensor information for producing a product. That is, the quality judgment system according to the present invention is to determine whether the defective image by determining the image of the product in the process of producing the product, it should be linked with the facility. Therefore, information on these facilities must be entered.

In addition, in the quality judgment system according to the present invention, various sensors can be installed to determine the product quality during the process. That is, not only an image sensor, but also sensors for sensing temperature, humidity, dust, and the like may be installed. In this case, information on these various sensors is input in advance.

In addition, the information on the facility and the information on the various sensors should be matched with each other.

In the inspection condition setting step (S120), it is set how to inspect the product. Here, it is common to set the conditions in time or in terms of quantity. That is, for example, it is possible to set a quantitative inspection method to execute the inspection once every 10 seconds or once every minute or to inspect one of ten or twenty products.

The quality standard setting step (S130) is a process of setting a defective condition of the product. That is, it is a process of recognizing the good and bad state of the product by learning a large number of good and defective state of the product.

In the facility control method setting step (S140), how will the facility control be performed when a defect occurs? And, when such a failure occurs, it is a process of setting to notify the failure state to the outside. In this case, when failure occurs continuously, the equipment is stopped, and the status is set to notify the remote user through text message (SMS) at the same time as notifying to the outside through the buzzer (Buzzer). That is, it is set to transmit a bad state to the user's mobile phone through the text message (SMS).

Here, the continuous failure state may be variously changed depending on the degree of quality requirements of the product or the user's will. In other words, if three defects are continuously reported, stop the equipment and notify the outside at the same time, or if three defects are continuously, only the function of notifying the outside is activated, and in case of ten consecutive defects Various settings are possible, such as to stop the equipment.

The product inspection step (S150), the image reading process (152) for taking an image of the product to read the image (Image) and the image editing process for editing the image read in the image reading process (152) (154) And an image storing process 156 for storing the image manipulated by the image editing process 154.

3 is a detailed process of the product inspection step (S150) is shown sequentially.

As shown in the drawing, the image reading process 152 of the product inspection step S150 is a process of reading an image of a product photographed by the photographing apparatus 200 into a BMP file. That is, a plurality of products are simultaneously input as one image, and at this time, the input is at a resolution of 1024 × 768.

And, the image editing process 154 of the product inspection step (S150), converts the image read by the image reading process 152 into a binary image, grasp the position of each product and at the same time cut out by region A first operation S154 ′ and a second operation S154 ″ for facilitating image determination by adjusting the brightness and contrast of the image passed through the first operation S154 ′.

The first operation S154 ′ is a crop process of cutting a plurality of products one by one as shown, and the second operation S154 ″ increases contrast by adjusting brightness and contrast. Contrast process.

The image storing process 156 of the product inspection step S150 is a process of storing the BMP image files which have undergone the image editing process 154, respectively. That is, as shown, the process of storing the image to each BMP image file having a resolution of 128 × 128.

4 is a cross-sectional view illustrating a configuration of the photographing apparatus 200 used in the image reading process 152.

As shown in the drawing, the photographing apparatus 200 captures an image of the support 220 which is fixedly installed on one side of the conveyor belt 210 to which the product is conveyed, and the product conveyed by the conveyor belt 210. The camera 230 to transmit the same, the lens 240 provided at the tip of the camera 230, and in the direction of the production line to provide the same illumination at all times within the shooting range of the camera 230 and to minimize shadows. It consists of a pair of lights 250, such as a bar (Bar) to be mounted.

The support 220 is installed at the upper end of the cradle 260, the conveyor belt 210 is inclined to the cradle 260. Therefore, when the product is transported along the conveyor belt 210, the photographing device 200 is to take an image of the product.

The camera 230 is configured to have a function of supporting a long-distance transmission protocol, and is installed on the support 220 so as to be able to be UP or DOWN. Thus, the user can arbitrarily adjust the height of the camera 230.

The illumination 250 is provided at a position spaced apart from the lens 240, and is mounted to the support 220 so as to be able to be UP or DOWN. That is, the illumination 250 is installed at a position spaced a predetermined distance below the lens 240, as shown in the form of a bar (Bar) and are spaced apart from each other in a pair. Therefore, when the product is positioned between the pair of lights 250, the photographing is performed by the camera 230.

In addition, the pair of lights 250 are mounted in a production line direction to be perpendicular to the conveyor belt 210. That is, the pair of lights 250 are respectively installed to cross the conveyor belt 210 back and forth (in FIG. 4). Accordingly, when the product is gradually moved from the left end of the conveyor belt 210 to the right end, the product is positioned between the pair of lights 250 at any moment (when viewed from above). The photographing is performed through 230.

In the product recognition step (S160), the product type is determined by the image of the product measured in the product inspection step (S150), and it is determined whether or not the input image product is defective according to the quality standard of each product. This product recognition step (S160) is composed of a total of four levels, each node (Node) is represented by one rectangle.

5 schematically shows the configuration of the product recognition step (S160).

As shown in the drawing, the product recognition step (S160) loads an image file of 128 × 128 pixels input in the product inspection step (S150) and reads it as a 16 × 16 node. A second level process (S164) of reading a first level process (S162) and a result of the conversion into a 16x16 node (Node) by the first level process (S162) as an 8x8 node (S164). And a third level process S166 of reading the result of the transformation into an 8x8 node by the second level process S164 as a 4x4 node, and the third level process. And a fourth level process S168 for recognizing the entire image area converted into 4x4 nodes at a time (S166).

In this case, as the level is increased, that is, the number of nodes decreases as the level is increased. That is, in the first level process S162 to the third level process S166, four nodes in the child node become one node in the parent node. In the fourth level process S168, a total of 16 nodes are combined into one node so that the entire image area can be recognized at once.

When the image is finally determined as described above, the type of the product is determined, and thus, by comparing the input image with the reference image of the corresponding product, it is determined whether the product having the input image is defective or good quality. This step is the product quality determination step (S170).

The subsequent action step (S180) is a process of processing a defect in a product, and takes action according to the contents set in the facility control method setting step (S140).

The subsequent action step (S180), the facility control process (S182) for stopping the equipment when the product defects occur continuously for more than a predetermined number of times, and generates an alarm when the product defects occur more than a predetermined number of times in a row. At the same time, an external notification process (S184) for transmitting a current situation to a remote user through a messenger (SMS).

In detail, as described in the facility control method setting step (S140), if a failure occurs continuously, the facility is stopped, and a text message (SMS) is simultaneously notified to the outside through a buzzer. Through the remote user will be notified. In other words, if three defects occur continuously, measures are taken such as stopping the equipment and informing the outside at the same time.

Meanwhile, the item standard information input step (S100), facility information input step (S110), inspection condition setting step (S120), quality standard setting step (S130), and facility control method setting step (S140) are performed simultaneously or in sequence. It is performed in turn regardless. In other words, the steps may be performed regardless of the order, and may be simultaneously performed.

6 is a main flowchart of an intelligent quality determination system based on HTM according to the present invention.

As shown in this, when the system is started, basic information about equipment and items is loaded (S300). In addition, the defect number C representing the quantity of the product found to be defective is set to zero (S310).

Next, it is determined whether an image is input (S320). If an image is not input, it is continuously detected whether an image is input, and if an image is input, the image processing step (S330) is performed. Passing The image processing step S330 is a step corresponding to the product inspection step S150 described above.

Then, it determines what the corresponding product is and matches the setting criteria of the corresponding product (S340). This process corresponds to the product recognition step (S160).

As such, when the input image is matched with the learning image of the corresponding product and compared, it is determined whether the product is defective (S350). In addition, this step (S350) corresponds to the product quality determination step (S170) described above.

As a result of the determination in step S350, if the product is a good product, the process returns to step S310 as shown.

On the other hand, if the product is defective, 1 is added to the defective number C (S360), and then it is determined whether the defective number C reaches the set reference number N (S370). That is, it is determined whether the user has reached the continuous failure reference number N set in the facility control method setting step S140 (S370).

As a result of the determination in step S370, when the continuous failure number C does not reach the reference number N, the flow returns to the above step S320.

On the other hand, when the continuous failure number (C) reaches the reference number (N), the subsequent action step (S180) proceeds (S380). That is, at this time, the facility is stopped according to the user's setting, and the text message (SMS) is transmitted to the user at the same time as the notification to the outside through the buzzer (Buzzer).

When the defective state is delivered to the user by the above process, the user directly checks the defective state of the product and checks the equipment.

The scope of the present invention is not limited to the above-exemplified embodiments, and many other modifications based on the present invention will be possible to those skilled in the art within the above technical scope.

1 is a diagram schematically showing a recognition function by an HTM-based recognition engine.

Figure 2 is a schematic block diagram showing the configuration of a preferred embodiment of the HTM based intelligent quality determination system according to the present invention.

3 is a view showing a detailed process of the product inspection step constituting the embodiment of the present invention.

Figure 4 is a front view showing the installation configuration of the photographing apparatus used in the product inspection step constituting an embodiment of the present invention.

5 is a view for explaining a product recognition step constituting an embodiment of the present invention.

6 is a flow chart showing a use state of the embodiment of the present invention.

Explanation of symbols on the main parts of the drawings

S100. Item standard information input step S110. Equipment information input step

S120. Inspection condition setting step S130. Quality standard setting stage

S140. Facility control method setting step S150. Product inspection stage

S160. Product recognition step S170. Product Quality Judgment Stage

S180. Follow-up Steps 200. Filming Equipment

210. Conveyor Belt 220. Support

230. Camera 240. Lens

250. Lights

Claims (5)

An item reference information input step (S100) of inputting information of a product having good quality; A facility information input step (S110) for inputting product production facility information for producing a product and sensor information for measuring a state of an ambient environment such as pressure or temperature; An inspection condition setting step (S120) of setting a condition in a timely or quantitative manner in which a product inspection is to be performed; A quality standard setting step (S130) for setting each of a plurality of products and how to make a criterion for determination of defective products and good products; Facility control method setting step (S140) of setting a control method of how to control the operation of the facility in case of a failure; An image reading process 152 for taking an image of a product and reading an image, an image editing process 154 for editing an image read in the image reading process 152, and an image editing process 154 A product inspection step (S150) of inspecting a product through an image storage process 156 of storing the image manipulated by; A product recognition step (S160) of recognizing a product image measured in the product inspection step (S150), and recognizing the type and correct product shape of the product; A product quality determination step (S170) of comparing the image finally recognized in the product recognition step (S160) with the reference image of the corresponding product to determine a defective product or a good product; If a product failure occurs by the product quality determination step (S170), the following steps for controlling the facility according to the control method set in the facility control method setting step (S140) at the same time to notify the fact to the outside (S180). HTM based intelligent quality judgment system, characterized in that it comprises a. According to claim 1, The product recognition step (S160), It consists of 4 levels in total, and each node is represented by one rectangle; A first level process (S162) of loading an image file of 128 × 128 pixels input in the product inspection step (S150) and reading it into a 16 × 16 node (Node); A second level process S164 of reading the result of the conversion into a 16x16 node by the first level process S162 as an 8x8 node; A third level process S166 of reading the result of the transformation into an 8x8 node by the second level process S164 as a 4x4 node; Intelligent quality based on HTM, characterized in that it comprises a fourth level process (S168) for recognizing the entire image area converted into a 4x4 node (Node) by the third level process (S166) at once. Judgment system. The method of claim 1, wherein the image reading process 152 of the product inspection step S150 is a process of reading an image of a product photographed by the photographing apparatus 200 into a BMP file; The image editing process 154 of the product inspection step (S150), The first operation process (S154 ') for converting the image read by the image reading process (152) into a binary image, grasping the outline position of the product, and cutting each region, and the first operation process (S154'). A second operation (S154 ″) for adjusting image brightness and contrast to facilitate image determination; Image storage process 156 of the product inspection step (S150), HMP-based intelligent quality determination system, characterized in that for storing; BMP image file having a resolution of 128 × 128 resolution of the image that passed through the image editing process (154). The method of claim 3, wherein the photographing apparatus 200, A support 220 fixedly installed at one side of the conveyor belt 210 to which the product is transferred; A camera (230) installed on the support (220) so as to be able to be upgraded (UP) and down (DOWN), and photographing and transmitting an image of a product conveyed by the conveyor belt (210); A lens 240 provided at the front end of the camera 230; It is provided at a position spaced apart from the lens 240, the support 220 is mounted to be up (UP) / down (DOWN), and always provides the same illumination within the shooting range of the camera 230 and the shadow HTM-based intelligent quality judgment system comprising a; a pair of lights (250) in the form of a bar (Bar) is mounted in the direction of the production line to minimize the. The method of any one of claims 1 to 4, wherein the follow-up step (S180), A facility control process (S182) for stopping the facility when a failure of the product occurs a predetermined number or more continuously; When the product defects occur continuously for a predetermined number of times, HTM, characterized in that it comprises an external notification process (S184) for transmitting the current situation to a remote user via a messenger (SMS) at the same time. HTM) based intelligent quality judgment system.

Images