KR20200073588A - A production line monitoring system using motion recognition of a workekr and a monitoring method using the same - Google Patents

Abstract

The present invention relates to a production line monitoring system using worker motion recognition and a production line monitoring method using the same. The production line monitoring system includes: a detection part installed in a workplace to obtain an image of the workplace; an image recorder connected with the detection part through a wire or wirelessly to record and store the image received from the detection part; a working environment server including working environment information of the workplace connected with the image recorder through a wired or wireless network; an image analysis server analyzing a repeated motion performed by a worker on a production process or element motions constituting the repeated motion; and a motion recognition server analyzing an image which is not stored in the image analysis server to analyze the conformity of the image with the repeated motion or element motions stored in the image analysis server. According to the present invention, the production line monitoring system is capable of reducing the loss rate of a production process caused by working motions on a working site and suggesting an optimal working environment scenario.

Description

A production line monitoring system using worker motion recognition and a method for monitoring a production line using the same{A PRODUCTION LINE MONITORING SYSTEM USING MOTION RECOGNITION OF A WORKEKR AND A MONITORING METHOD USING THE SAME}

The present invention relates to a production line monitoring system and a monitoring method using the same, and more specifically, to monitor a production line through an operator's motion recognition through AI, using a worker's motion recognition to significantly reduce the LOSS rate during the work process. It relates to a production line monitoring system and a method for monitoring a production line using the same.

Various methods have been used in the past to calculate the time it takes to produce a product in the manufacturing industry. A simple measurement method using a stopwatch and an analysis method using a spreadsheet were used. Recently, the element motion analysis using a recorded video and a method using the same have been generally used. By using the results of this analysis, the results of LOB (Line of Balance), MMC, MAC, etc. have been derived and used for the purpose of improving productivity and reducing costs through this.

Recently, the interest and importance of smart factories are increasing worldwide, and factory automation is recognized as the latest trend, mainly in Germany, the United States, and Japan. Korea is also investing a lot of money to promote the smart factory, and MES-based facility-based investment is being made, and the process of analyzing and quantifying productivity according to the operation result of the facility is focused. However, solving the problem centered on mechanical factors is a part of many variables that can occur, and rather, productivity is determined by human factors. In order to solve this problem, productivity is measured through the analysis of worker's repetitive work, but since this is a result of analysis for a short period of time, the effect of various production environment variables in the actual continuous and continuous production process is found. It is not easy to do.

Therefore, it is necessary to analyze the cause of the actual waste through the analysis of the effect on the worker's production, and after completing the research to solve this problem, the present invention was completed.

1. Registered Patent Publication No. 10-1535804 (2015.07.06.) 2. Registered Patent Publication No. 10-0453699 (2004.10.11.)

The present invention is to provide a production line monitoring system using operator motion recognition and a method for monitoring a production line using the same.

An aspect of the present invention is installed in a workplace, a detection unit for acquiring an image of a workplace, an image recording device for recording and storing an image received from the detection unit by wired or wireless connection with the detection unit, an image recording device and a wired or wireless network. A work environment server including work environment information of a connected workplace, a video analysis server that analyzes repetitive actions performed by a worker in a production process or elemental motions constituting a repetitive action, and a video that is not stored in a video analysis server to analyze the video It may be a production line monitoring system using operator motion recognition, including a motion recognition server that analyzes the degree of repetition or element motion stored in the analysis server.

The detection unit may include a CCD (Charge-Coupled Device) camera or a CMOS camera.

The work environment server may include one or more types of information selected from the group consisting of information on the workplace and types of items to be produced, BOM, input worker information, and production planning information.

In the image analysis server, a work environment selection process for inputting the work environment of the work environment server, a process for defining element motions for defining in detail what actions the repetitive action is made of, and the action used for production The process of dividing and inputting the time-added value activity (Value Added) and non-value activity (Non Value Added) not related to production may be performed.

In another aspect of the present invention, in a production line monitoring method through operator motion recognition, a step of inputting a work environment, a step of shooting a motion image, and element operation of a work motion using a video to create basic information of AI Analyze according to the stage, analyze the subsequent motion through the motion recognition AI server, check the content and accuracy of work motion analysis, monitor and monitor the field situation to analyze and apply specific patterns, and connect the motion recognition result and production data And it may be a method for monitoring the production line through operator motion recognition, including the step of analyzing.

According to the present invention, it is possible to provide a production line monitoring system using operator motion recognition and a method for monitoring production using the same.

In addition, it is possible to reduce the loss rate due to work operation at the work site during the manufacturing process and to present an optimal work environment scenario.

1 is a schematic diagram schematically showing the configuration of a production line monitoring system using operator motion recognition according to an aspect of the present invention.
2 is a schematic diagram sequentially showing a method of monitoring a production process using a production line monitoring system according to an aspect of the present invention.
FIG. 3 is a schematic diagram schematically showing the procedure of analyzing the operation of FIG. 2.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. Embodiments of the present invention may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. Embodiments of the present invention are provided to more fully describe the present invention to those skilled in the art. Accordingly, the shape and size of elements in the drawings may be exaggerated for a clearer description, and elements indicated by the same reference numerals in the drawings are the same elements.

Terms such as “... unit” and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software. Also, when a part of the specification is said to be “connected” to another part, this includes not only “directly connected” but also “electrically connected” with another element in between.

The present invention relates to a production line monitoring system using operator motion recognition and a method for monitoring a production line using the same.

1 schematically shows a configuration of a production line monitoring system using operator motion recognition according to an aspect of the present invention. 2 sequentially shows a method of monitoring a production process using a production line monitoring system according to an aspect of the present invention. FIG. 3 schematically shows the procedure for analyzing the operation of FIG. 2.

Referring to FIG. 1, a production line monitoring system using operator motion recognition according to an aspect of the present invention includes a detection unit 100 included in each workshop 10 and an image recording device 20 connected to the detection unit by wire or wirelessly. ), a work environment server 40, an image analysis server 50, and a motion recognition server 60 connected through an image recording device and a wired or wireless network 30.

The workshop 10 may be domestic or overseas, and refers to a space that can be connected to the video recording device 20 wirelessly or by wire. The workplace 10 may have a detection unit 100 connected to the video recording device 20 or serving as the video recording device 20 itself.

The work images collected through the detector 100 installed in the workshop 10 are transmitted in real time to the work environment server 40 and the motion recognition server 50 through the network 30 to process work analysis through motion recognition. It can be used as basic information.

The number of workers put into the workshop 10 may be not one. In this case, motion recognition for each worker may be set.

The detection unit 100 may include a camera for photographing the working status of the workplace. For example, the camera may be a CCD (Charge-Coupled Device) camera, a CMOS type camera, and the like, but is not limited thereto and may refer to all equipment for shooting.

The image recording device 20 refers to a device capable of storing a work image received from the detection unit 100 for a certain period of time. The image recording device 20 may be included in the detection unit 100 or may be operated independently.

The server may include a work environment server, an image analysis server, and a motion recognition server.

The work environment server 40 may include information such as information of the workshop 10 and types of items produced, BOM, input worker information, and production planning information. Production planning information may include information on the start time of production, the end time of production, items to be produced, and input workers. In addition, the production planning information may include production environment such as temperature, humidity and pressure that can be produced according to the manufacturing method, replacement time of molds, models and jigs, and conveyor speed.

The functions of each of the work environment server 40, the image analysis server 50, and the motion recognition server 60 can perform all functions on one server according to the level affecting the performance of the system, and also each function If expansion is needed, more than one server can be operated to perform one function.

The image analysis server 50 is a server that defines repetitive actions or element actions constituting repetitive actions that occur while an operator proceeds with production. The image analysis server 50 defines repetitive or elemental motions, and inputs information about the start time and end time of each repetitive motion in the image. In this process, a process of extracting or storing an image of the image recording device 20 used for the purpose of analysis may be included. In addition, information on the work environment, item type, BOM, and production plan information stored in the work environment server 40 may be input to the information of the repetitive action or element action.

The motion recognition server 60 is a server for performing work motion analysis (S300), subsequent motion recognition (S400), and whether or not an unusual pattern is generated (S510). The motion recognition server 60 recognizes the posture of an operator appearing in an image recorded in the image recording device 20 and stores the posture change information of the operator. The motion recognition server 60 analyzes other images in the image recording device 20 that are not stored in the image analysis server 50, and repeats or elemental motion in which the posture change information of the worker is stored in the image analysis server 50. It analyzes the degree of coincidence with and displays the accuracy of the corresponding repetitive or elemental motion. Here, the information of the image analysis server 50 is filtered based on the production plan information by reflecting the time of the image recording device 20 and the time of the production plan information stored in the work environment server 40 to increase the accuracy of image recognition. Can be used for

In the image analysis server 50, a work environment selection process (S310) of inputting to which work environment the corresponding video corresponds to the work environment server (S310), and element motion definition to define in detail what action the repetitive action is made of ( S320) and a process in which the corresponding operation is divided into a value activity that is used for production and a non-value activity that is not related to production.

2 shows a flow chart for a method of monitoring a production line using a production line monitoring system to which AI worker motion recognition is applied.

Referring to FIG. 2, a production line monitoring method through operator motion recognition according to another aspect of the present invention includes inputting a work environment (S100), photographing a motion image (S200), and preparing basic information of AI In order to analyze the work motion according to the element motion in order to use the video (S300), analyze the subsequent motion through the motion recognition AI server to check the content and accuracy of the work motion analysis (S400), and monitor the field situation It may include the step of analyzing and applying the unusual pattern (S500), and the step of connecting and analyzing the motion recognition result and the production data (S600).

First, a working environment may be input (S100). It is possible to input information such as information of a workplace contained in the work environment server 40, types of items produced, BOM, input worker information, and production planning information.

Next, an operation image may be photographed (S200 ). The shooting can be performed using a camera, and a CCD (Charge-Coupled Device) camera, a CMOS camera, or the like can be used as the camera.

Next, in order to create basic information of the AI, the motion of the work may be analyzed according to the motion of the element using a video (S300).

After the AI recognizes the motion, the image analysis server 50 may input information such as a start time, an end time, and a job name for the repetitive work stored in the image recording device 20. In addition, for detailed analysis of repetitive motions, the name of each detailed motion of the operator can be designated, and the detailed motions can be arbitrarily assigned according to the motion of joints such as fingers, wrists, elbows, shoulders, waists, and legs. , It can be classified arbitrarily, including the complex movement of two or more joints.

In the image analysis server 50, a work environment selection process (S310) of inputting to which work environment the corresponding video corresponds to the work environment server (S310), and element motion definition to define in detail what action the repetitive action is made of ( S320) and a process in which the corresponding operation is divided into a value activity that is used for production and a non-value activity that is not related to production.

In performing the work motion analysis (S300), when the repetitive motion needs to recognize and manage the motion in detail, the work environment server 40 sets the repetitive motion to include the element motion to analyze the video analysis server ( It may include the step of classifying the element motion from 50) and inputting the result value, and assigning a score to the recognition accuracy of each element motion corresponding to the repetitive motion in the motion recognition server 60.

Next, by analyzing the subsequent motion through the motion recognition AI server, it is possible to check the work motion analysis content and accuracy (S400).

Based on the information obtained from the image analysis server 50 and the motion recognition server 60, the motion recognition server 60 recognizes the operator's motion from the images recorded in the image recording device 20 and recognizes the existing information and By comparing repetitive motions, it is possible to judge the degree of correspondence of motions and assign scores according to their accuracy.

In addition, if the score exceeds the target score set arbitrarily based on the assigned score, the job recognition server 60 may determine that the result of the job recognition is suitable for monitoring (S500), and reaches the target score set arbitrarily If not, additional job analysis may be performed in the job analysis server 50 (S300).

In addition, when the element motion recognized by the motion recognition server 60 exceeds the target score set arbitrarily, the motion recognition server 60 determines that the element motion recognition result is suitable for monitoring (S500). Alternatively, if the target score set arbitrarily is not reached, additional work analysis may be performed in the work analysis server 50 (S300).

Next, the singular pattern can be analyzed and applied by monitoring the site situation (S500).

The motion recognition server 60 monitors in real time what repetition work is being performed on the current video based on the video recorded in the video recording device 20, and recognizes an existing accumulated pattern and a pattern of a different type to analyze the video. It can be used to increase the recognition rate of the motion recognition server 60 by accumulating data by analyzing the motion of the corresponding image in the image analysis server 50 by transmitting it to (50).

By continuously analyzing the image of the video recording device 20 in the motion recognition server 60, it is possible to check whether the current operator's motion belongs to any repetitive motion or detailed motion, and when the recognition rate of the repetitive motion is lower than a predetermined criterion It can be classified as an unusual pattern and transmitted to the image analysis server 50.

The image analysis server 50 analyzes the corresponding image and checks whether it belongs to a repetitive operation or a detailed operation in the existing data, and if it corresponds to the existing data, analyzes the repetitive operation and accumulates additional data. If the content is different from the existing data, it can be classified as a special feature to define what type of motion the corresponding video is, and define the value to be input when the type of motion is recognized.

Next, as a video of the video recording device 20, the time at which the corresponding task analyzed by the motion recognition server 60 was started, the time at which the task was analyzed, the time length of the repetitive task, and the time information of the specific pattern by type are collected and produced. The status can be updated in the work environment server 40 (S600).

Based on the collected information, the work environment server 40 can analyze the current status of the production plan, the efficiency level of the repetitive work, and the time status caused by the unusual pattern.

Next, in the motion recognition server 60 based on the result of the connection of the worker motion type recognized by the motion recognition server 60 with the same image as the information accumulated in the video analysis server 50 in the video recording device 20 When the image analysis server 50 recognizes an image that has not been analyzed, the image analysis server 50 and the motion recognition server 60 express similar levels as previously stored information as a computed value such as a percentage or a number, Data on the analysis result may be transmitted to the user 70 through the network 30 (S700).

In the present invention, the operation of the worker can be largely classified as a move operation, an end operation, and other operations.

The move action is to move the product to another location, such as a move action using fingers, a move action using palms and fingers from the wrist, a move action using lower arms, hands and fingers, upper and lower arms, palms And a moving action using fingers and a moving action that stimulates the unfolded arm and shoulder muscles.

The end action is that the action is made in the end, and the end action can be divided into a hold action and a release action. The gripping motion can be classified into a gripping motion that requires simple attention to an object, a gripping motion that is simply grasping, a gripping motion that involves correcting the eye, attention, and movement more than once, and a gripping motion that requires a high degree of attention using fingertip joints. have. The releasing action is categorized into the simplest releasing action that requires no attention, the releasing action in which the hand places the object through one or more modifications, the releasing action that requires sensibly possible attention, and the releasing action that puts the eye in the correct position. You can.

Other movements include heavy movements that carry heavy objects, viewing movements when moving and focusing the eyes, judgment movements that require instant judgment, pressing operations with pressures of 2 kg or less (hand movements), pressing with pressures of 2 kg or more Motion (whole body), walking or rotating one step, re-grasping to catch small objects, re-grabbing to catch large objects, foot to tread or release the heel on the floor, circular movement of the object It can be categorized into rotating motion, bending and standing up again, sequential bending and standing up again, sitting and sitting on a chair, reciprocating, attention and danger.

The terminology used in the present invention is for describing specific embodiments, and is not intended to limit the present invention. Singular expressions should be considered to include plural meanings, unless the context is clear. The terms "include" or "have" mean that there are features, numbers, steps, actions, elements, or combinations thereof described in the specification, and are not intended to exclude them. The present invention is not limited by the above-described embodiments and the accompanying drawings, but is intended to be limited by the appended claims. Accordingly, various forms of substitution, modification, and modification will be possible by those skilled in the art without departing from the technical spirit of the present invention as set forth in the claims, and it should be considered that this also belongs to the scope of the present invention. something to do.

10: workshop 20: video recording device
30: Network 40: Work Environment Server
50: video analysis server 60: motion recognition server
70: user 100: detection unit

Claims (5)

A detection unit installed in a workplace to acquire an image of the workplace;
An image recording device connected to the detection unit by wire or wireless to record and store the image received from the detection unit;
A work environment server including work environment information of a workplace connected to the video recording device through a wired or wireless network;
An image analysis server that analyzes repetitive actions performed by a worker in a production process or elemental motions constituting repetitive actions; And
A motion recognition server that analyzes an image not stored in the image analysis server and analyzes the degree of repetition or elemental motion stored in the image analysis server;
Including, production line monitoring system using operator motion recognition.
According to claim 1,
The detection unit includes a CCD (Charge-Coupled Device) camera or a CMOS camera, a production line monitoring system using operator motion recognition.
According to claim 1,
The work environment server is a production line monitoring system using worker motion recognition, which includes one or more types of information selected from the group consisting of information on the workplace and types of items produced, BOM, input worker information, and production planning information.
According to claim 1,
In the image analysis server, a work environment selection process for inputting a work environment of the work environment server, a process for defining element motions for defining in detail what motions the repetitive motions are made, and the corresponding motions for production. A production line monitoring system using worker motion recognition, where the process of dividing and inputting the used time, value added and non-value added, not related to production, takes place.
In the production line monitoring method through operator motion recognition,
Inputting a working environment;
Photographing a motion image;
Analyzing the operation of the work according to the element operation using a video to create basic information of the AI;
Analyzing the subsequent motion through the motion recognition AI server to check the content and accuracy of the work motion analysis;
Analyzing and applying singular patterns by monitoring the site situation; And
Connecting and analyzing motion recognition results and production data;
Including, production line monitoring method through operator motion recognition.

Images