KR102642975B1 - Method and system for job guiding through job guide device - Google Patents

Abstract

The present invention relates to a work guidance method through a work guide device, which is performed by a system, wherein (a) a work guide device mounted on the worker's head photographs the worker's front view and transmits the captured image to an edge server; collecting fastening state data through a fastening device and transmitting the collected fastening state data to the edge server; (b) the edge server analyzing the image received from the work guide device through a previously learned work analysis model to determine the type of work the worker is working on and whether there is a work error; and (c) providing, through the work guide device, work instruction information corresponding to the identified work type and whether or not there is a work error, by the edge server; Including, the work instruction information is at least one of guidance information for the current task, information about errors, and guidance information for the next task, and the task analysis model includes the operator's motion images and tasks for each task type. It relates to a work guidance method using a work guide device, which is a model that is learned using images of objects and work tools as learning data and outputs the type of work and whether or not there are work errors from the input images.

Description

Job guide method and system through job guide device {METHOD AND SYSTEM FOR JOB GUIDING THROUGH JOB GUIDE DEVICE}

The present invention relates to a work guide method and system through a work guide device. More specifically, the present invention relates to a work guide device mounted on the worker's head, which recognizes the work currently being performed by the worker and provides corresponding work instructions and frequent work instructions. A method and system for providing a work guide device for performing inspection.

According to a study by the National Statistical Office, the domestic manufacturing industry is expected to grow at an average annual growth rate of 41%, increasing 16 times in size over the next four years from approximately 22.4 trillion won in 2020 to approximately 360 trillion won in 2024. Currently, there are approximately 25,870 companies in the domestic manufacturing industry. It was found that businesses were participating and about 1,316,000 workers were engaged.

As for the manufacturing industry, assembly and processing industries such as mechanical equipment, office equipment, electrical machinery, video communication equipment, precision equipment, optical equipment, automobiles, assembled metal, and transportation equipment account for more than 54% of the total.

As mentioned above, the conventional assembly and processing manufacturing industry is not befitting the status of a large industry, and as multiple workers assemble and process various heterogeneous parts, defects inevitably occur during production, and in order to solve this problem, There is a problem in that self-inspection must be conducted immediately after production or during the production process.

The more frequently such self-inspection is conducted, the lower the defect rate of manufactured products, but the production quantity may decrease due to loss of working time, which may actually reduce production efficiency.

Therefore, in order to solve the problems of the prior art, a method is provided to provide a work guide device that recognizes the work currently being performed by the worker through a work guide device mounted on the worker's head and performs corresponding work instructions and frequent inspections. and systems are needed.

The present invention is intended to solve the problems of the prior art described above, and aims to provide a method and system for guiding work through a work guide device.

Through this, rather than conducting a separate independent inspection of the work being performed by the worker after the worker stops working, the inspection is conducted simultaneously while the work is being performed, and information about the inspection is provided through a work guide device. The purpose is to maximize production efficiency.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood from the description below.

As a technical means for achieving the above-described technical problem, a work guiding method through a work guide device performed by a system according to an embodiment of the present invention includes (a) a work guide device mounted on the worker's head; Photographing the worker's front view, transmitting the captured image to an edge server, collecting fastening state data through a fastening device, and transmitting the collected fastening state data to the edge server; (b) the edge server analyzing the image received from the work guide device through a previously learned work analysis model to determine the type of work the worker is working on and whether there is a work error; and (c) providing, through the work guide device, work instruction information corresponding to the identified work type and whether or not there is a work error, by the edge server; Including, the work instruction information is at least one of guidance information for the current task, information about errors, and guidance information for the next task, and the task analysis model includes the operator's motion images and tasks for each task type. It may be a model that is learned using images of objects and work tools as learning data, and outputs the type of work and whether there are work errors from the input images.

In addition, in step (a), the fastening device is rotated according to a control signal while in contact with the fastening screw, and the fastening screw is rotated to fasten the fastening screw to the work object, and the fastening device is rotated according to the unit time generated during the fastening operation. At least one of the rotation speed (RPM) and torque value of the motor and the fastening screw may be collected as fastening state data and provided to the edge server.

In addition, step (b) includes: (b-1) dividing the image received by the edge server from the work guide device into a work object, work tool, worker's body, and other objects according to an image analysis technique; (b-2) The images of the classified work objects and work tools are compared with the previously saved images for each work type and work progress, and if they match more than a preset value, the current work progress is judged as the above work progress, and the classified Comparing the image of the work object and the fastening device with a pre-stored image for each work error, and if the image matches a preset value or more, determining whether there is a work error in the current work; and (b-3) when the images of the classified work and work tools match the pre-stored work progress image and a pre-set value or more, determining that the work progress is the pre-stored work progress. can do.

In addition, in the step (b-2), based on the RPM and torque values of the fastening state data collected from the fastening device, abnormal fastening of the fastening screw and the work object is detected by comparing it with a previously stored graph for each type of abnormal fastening. By doing so, it may be possible to determine whether there is a work error.

In addition, the work guide device includes a support portion that rests on the bridge of the worker's nose and supports the main body portion; The main body portion has a predetermined thickness and is formed in a semicircular shape at a predetermined distance above the worker's eyebrows, and is connected to an upper side of the support portion; a leg portion extending from the main body portion and fixed to the worker's ear, and including a camera device attached to the outer side of the worker's head to capture the front view of the worker; A bone conduction speaker device that outputs a voice corresponding to work instruction information and a control signal signal according to a control signal may be built-in.

In addition, the work guide device is formed to be spaced a predetermined distance away from the operator's eyeballs so as not to block the captured image of the camera device, extends from the lower side of the main body, and is made of a transparent or translucent material so as not to block the operator's field of view. It may further include a glass portion composed of.

In addition, the work guide device includes a main body portion that has a predetermined thickness and is formed and supported in a semicircular shape in contact with the back of the worker's head; a support portion that has a predetermined thickness, is in contact with the worker's temporal portion, extends from the main body portion above the worker's ear to the worker's temple, and rests on the worker's ear to support the main body portion; The support unit includes a camera device attached to the outer side of the worker's head to photograph the worker's front view, and the main body unit has a voice and a voice corresponding to work instruction information according to a control signal received from the edge server on the inside. A bone conduction speaker device that outputs a control signal sound may be built in.

In addition, in step (c), the work guide device transmits work instruction information corresponding to the work type, work progress, and work error received from the edge server through the bone conduction speaker device inside the work guide device. It may be provided as a voice corresponding to work instruction information and a voice control signal.

In addition, step (c) may be to have the work guide device provide a work error signal including at least one of sound, vibration, and image when it is determined that the current work order is incorrect or a work error has occurred. there is.

In addition, in step (c), the work guide device displays work instruction information corresponding to the work type and work progress received from the edge server by overlapping the operator's field of view on the glass portion of the work guide device. It may be provided through images.

In addition, step (c) determines the type of work the worker is working on and whether there is a work error, and when it is determined that the work has been completed without error, the preset next work instruction information is provided to the work guide device, When the worker's hand part in the image received from the camera device of the work guide device is recognized and a control hand signal is extracted from the recognized hand part, the work instruction information corresponding to the extracted control hand signal is sent to the next work instruction information set above. First of all, it may be provided as a work guide device.

Additionally, (d) the edge server provides the main server with at least one of image data, number of task errors, number of gesture inputs, task yield, and task time for a plurality of tasks, and the main server provides each task. It may further include the step of automatically generating an analysis report for each worker who performed it and providing it to the administrator terminal.

A work guide system using a work guide device according to an embodiment of the present invention includes a head-mounted bone conduction work guide device that photographs the front view of a worker and transmits the photographed image to a server; A fastening device that collects fastening state data and transmits the collected fastening state data to an edge server; And the image and fastening state data received from the work guide device and the fastening device are analyzed through a previously learned work analysis model to determine the type of work the worker is working on and whether there is a work error, and the identified work type and task. An edge server that provides at least one of work instruction information about progress and error status and guidance information about the next task through the work guide device, wherein the work instruction information is information about whether the current task has an error. and guidance information for the next task, and the task analysis model is learned using the operator's motion images for each task type and images of work objects and work tools as learning data, and calculates the task type and task from the input images. It may be a model that outputs whether there is a task error.

The present invention provides a work guide method and system through a work guide device, so as to conduct frequent inspections of the work being performed by the worker while working, and provide information about the inspection to the work guide device for production. Efficiency can be maximized.

In addition, the work guide device identifies the work object and task currently being worked on, and provides work instruction information corresponding to the identified work through intuitive means such as voice, video, or vibration and light, helping workers who are not familiar with the work. We can support you to have a certain level of work ability.

Figure 1 is an example of a work guide system using a work guide device according to an embodiment of the present invention.
Figure 2 is a block diagram showing the internal configuration of an edge server according to an embodiment of the present invention.
Figure 3 is a conceptual diagram of an image and image processing algorithm according to an embodiment of the present invention.
Figure 4 is an illustration of a work guide device according to an embodiment of the present invention.
Figure 5 is an example of operator gesture recognition of a work guide device according to an embodiment of the present invention.
Figure 6 is a flowchart of the execution order of a work guide method using a work guide device according to an embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" with another element in between. . Additionally, when a part "includes" a certain component, this means that it may further include other components rather than excluding other components, unless specifically stated to the contrary.

In this specification, 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware. Meanwhile, '~ part' is not limited to software or hardware, and '~ part' may be configured to reside in an addressable storage medium or may be configured to reproduce one or more processors. Therefore, as an example, '~ part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card.

The “terminal” mentioned below may be implemented as a computer or portable terminal that can connect to a server or other terminal through a network. Here, the computer is, for example, a laptop equipped with a web browser, a desktop, a laptop, a VR HMD (e.g., HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.), etc. may include. Here, VR HMD is for PC (e.g. HTC VIVE, Oculus Rift, FOVE, Deepon, etc.), mobile (e.g. GearVR, DayDream, Storm Magic, Google Cardboard, etc.), and console (PSVR). Includes independently implemented Stand Alone models (e.g. Deepon, PICO, etc.). Portable terminals are, for example, wireless communication devices that ensure portability and mobility, including smart phones, tablet PCs, and wearable devices, as well as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, and ultrasonic devices. , may include various devices equipped with communication modules such as infrared, WiFi, and LiFi. In addition, “network” refers to a connection structure that allows information exchange between nodes such as terminals and servers, including a local area network (LAN), a wide area network (WAN), and the Internet. (WWW: World Wide Web), wired and wireless data communication network, telephone network, wired and wireless television communication network, etc. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, and ultrasound. This includes, but is not limited to, communication, Visible Light Communication (VLC), LiFi, etc.

The present invention relates to a work guidance method through a work guide device, which recognizes the work currently being performed by the worker through a work guide device mounted on the worker's head and provides corresponding work instructions through various means. , This is a technology that automatically performs frequent inspections without stopping work during work.

With reference to Figure 1, a work guide system using a work guide device according to an embodiment of the present invention will be described.

A work guide system using a work guide device according to an embodiment of the present invention includes an edge server 100, a work guide device 200, a main server 300, and an administrator terminal 400.

The edge server 100 is installed in a workspace where work is performed, such as a workbench or workstation, and is connected and communicates wired and wirelessly with the work guide device 200 and the main server 300, and provides work instructions to the work guide device 200. It may provide information and perform frequent inspection of the currently performed work based on data received from the work guide device 200.

The main server 300 can be implemented in the form of a normal server structure and a cloud server, and is responsible for automatically generating an analysis report based on a plurality of data received from the edge server 100 and performing tasks or self-inspection. It may be provided to the administrator terminal 400, which is a user terminal owned by personnel.

Referring to Figure 2, the edge server 100 and the main server 300 according to an embodiment of the present invention include a memory storing a program (or application) that performs a work guidance method through the work guide device 200, and It may be configured to include a processor that executes the above program. Here, the processor can perform various functions depending on the execution of the program stored in the memory.

Next, the work guide device 200 may be implemented in various embodiments, but in the preferred embodiment, as shown in FIG. 4, a support portion 210 that rests on the bridge of the worker's nose and supports the main body 220, A main body portion 220 that has a thickness of and is formed in a semicircular shape at a predetermined distance above the worker's eyebrows and is connected to the upper side of the support portion 210, is formed to extend from the main body portion 220 and is fixed on the worker's ears, It may be configured in the form of glasses or goggle glasses without lenses, including a leg portion 230 that is attached to the outer side of the worker's head and includes a camera device 231 that photographs the worker's front view.

According to another embodiment of the present invention, the work guide device 200 is formed to be spaced a predetermined distance away from the operator's eyeballs so as not to block the captured image of the camera device 231, and the work guide device 200 is formed at a predetermined distance from the operator's eyes. It extends from the lower side and may further include a glass portion made of a transparent or translucent material so as not to block the operator's field of view. In this embodiment, the glass portion includes a lens that supplements the worker's visual ability, as well as a front view of the worker. It may be an electronic display that provides AR content.

Additionally, the work guide device 200 may be mounted on the back of the worker's head in the form of bone conduction earphones, rather than in the front direction of the worker's field of vision as described above.

In this embodiment, the work guide device 200 has a main body portion (not shown) that has a predetermined thickness and is in contact with the back of the worker's head and is formed and supported in a semicircular shape, and has a predetermined thickness and is in contact with the worker's temporal part to support the worker. It extends from the main body above the ears to the worker's temple, and is attached to a support part (not shown) that rests on the worker's ears and supports the main body, and a camera attached to the support part (not shown) in the outer direction of the worker's head to capture the worker's front view. It may also be configured to include a device (not shown).

In addition, in the above-described embodiments, the work guide device 200 includes a bone conduction speaker that outputs a voice corresponding to the work instruction information and a control signal voice according to the control signal received from the edge server 100 inside the main body. The device may be built-in.

Hereinafter, a detailed description will be given of the work guidance method using the work guide device 200 according to an embodiment of the present invention, which is performed by a system composed of the above-described elements.

First, the work guide device 200 mounted on the worker's head captures the worker's front view and transmits the captured image to the edge server 100, and collects the fastening state data through the fastening device. It is transmitted to the edge server 100.

At this time, the fastening state data is data generated when the fastening device is in contact with the fastening screw and rotates the fastening screw according to a control signal input by the operator to fasten the fastening screw to the work object, which occurs during the fastening operation. It may include at least one of the rotation speed (RPM) and torque value of the motor and fastening screw of the fastening device per unit time.

Referring to Figure 3, the edge server 100 analyzes the image received from the work guide device 200 through a previously learned task analysis model to determine the type of task the worker is working on and whether there is a task error.

At this time, the task analysis model of the edge server 100 is based on the CNN (Convolutional Neural Network) technique and can recognize work objects in specific images, determine work order, and detect work errors, and allow workers to control As an artificial intelligence learning algorithm that recognizes a given gesture, learning may have been completed in advance before implementation of the present invention.

Additionally, the task analysis model may be a model that is learned using images of the worker's movements for each type of task and images of work objects and work tools as learning data, and outputs the type of task and whether or not there is a task error from the input images.

To this end, the edge server 100 first divides the image received from the work guide device 200 into a work object, work tool, worker's body, and other objects according to an image analysis technique.

Image analysis techniques that can be used in the present invention include an object detection technique that can detect a specific object in a still image or video, and a YOLO (You Only Look) technique that predicts the object present in the image and the location of the object. Once) technique and a two-stage object detection technique using Faster RCNN can be used.

In addition to the above-described image analysis techniques, various image analysis techniques may also be used as embodiments of the present invention, but since they are prior art, they are not discussed in detail in this specification.

The edge server 100 compares the images of the classified work objects and work tools with the images for each work type and work progress previously stored in the DB or memory of the edge server 100, and if they match more than a preset value, the current work progresses. judge the degree

In addition, the images of the classified work objects and fastening devices are compared with the previously stored images for each work error, and if they match more than a preset value, work errors in the current work are identified.

For example, assuming that a worker is performing the task of attaching tape to a box and sealing it, the edge server 100 records the box, the tape, and the worker from the camera device 231 of the work guide device 200 equipped by the worker. You receive an image containing a body part (in most cases, both hands).

The edge server 100 analyzes the received image according to an image analysis technique, distinguishes the box, tape, and worker's hand in the image, and compares the classified image with the previously stored work type, image, and error status image. .

If the worker attaches tape to the box, but there is a part that cannot be sealed by mistake, the edge server 100 of the present invention already stores an image of the error in case of failure to seal, so it responds to the corresponding task. It can be determined that an error has occurred.

According to another embodiment of the present invention, when the type of work is fastening parts to a work object, the edge server 100 based on the RPM and torque values of fastening state data collected from the fastening device, By detecting the abnormal fastening of the fastening screw and the work object by comparing it with a previously stored graph for each abnormal fastening type, it is possible to determine whether there is a work error.

For example, assuming that the task of fastening screws to the wheel of a car is performed, the fastener that performs the task is connected to the edge server 100 wired or wirelessly, and performs the task through the edge server 100. It transmits fastening status data including the RPM and torque values that occurred when the connection was made.

The edge server 100 converts the received fastening status data into a graph and detects it by comparing the graph with graphs for each abnormal fastening type stored in the DB or memory.

At this time, when the images of the work and work tools classified as described above match the pre-stored work progress image and a pre-set value, the edge server 100 determines that the work progress is the pre-stored work progress. can do.

Therefore, when work is currently in progress, that is, when screws are still being fastened to the car wheel, considering that the work has not yet been completed, comparison with various preset images is not performed but is postponed and the fastening work is completed. Afterwards, it may be possible to detect whether there is an error.

Based on the type of work, work progress, and whether or not there are work errors identified through the above-described process, the edge server 100 may provide corresponding work instruction information through the work guide device 200.

Here, the work instruction information may be at least one of guidance information for the current work, information about errors, and guidance information for the next work.

For example, assuming that the task currently being performed by the worker is the task of dividing the work object into four parts, the edge server 100 provides a work instruction with the guidance information “Divide the work object into four parts” as guidance information for the current task. It can be provided as information.

In the same example, information about an error such as “the work object is not separated correctly” may be provided as work instruction information.

This means of providing work instruction information may be implemented in various embodiments, but as a representative example, work instruction information may be provided as a voice corresponding to the work instruction information through a bone conduction speaker device inside the work guide device 200.

At this time, if it is determined that the order of the current task is incorrect or a task error has occurred, a task error signal including at least one of sound, vibration, and image may be provided.

In addition, according to another embodiment of the present invention, when the work guide device 200 includes a glass part made of a transparent or translucent material so as not to block the operator's view, work instructions corresponding to the type of work and work progress are provided. Information can be provided through an AR image displayed on the glass portion of the work guide device 200 overlapping with the worker's field of view.

The work instruction information provided through the work guide device 200 determines the type of work the worker is working on and whether there is a work error, and if it is determined that the work has been completed without error, the work instruction information is sent to the work guide device ( 200).

To this end, the edge server 100 may store work instruction information preset for each detailed task and detailed process for a plurality of tasks in a DB or memory.

In addition, according to an embodiment of the present invention, the edge server 100 recognizes a gesture through a part of the worker's hand recognized in the image received from the camera device 231 of the work guide device 200, and detects the corresponding gesture. The corresponding operation can be performed.

When a control hand signal is extracted from a recognized hand part, the edge server 100 provides work instruction information corresponding to the extracted control hand signal, and the work instruction information triggered by the control hand signal is converted to the next work instruction information or error set. This may be given priority over work instruction information provided.

Referring to Figure 5, a gesture according to an embodiment of the present invention, that is, a control signal, may be data recorded as an image or video indicating that a specific action is performed through a body part of the worker, but in a preferred embodiment, the gesture is performed using a body part of the worker. It may be to utilize the area.

As shown, if the worker extends all fingers except the thumb, extends the hand, and waves horizontally to the left or right, shaking to the left provides previous work instruction information, and shaking to the right provides information on previous work instructions. The following work order information can be provided:

Additionally, if the worker clenches his fist with his thumb outward, he can immediately stop and skip the work instruction information currently being provided.

At this time, according to another embodiment of the present invention, if the worker is left-handed, the right hand is set as the hand for extracting the worker's control hand signal, and if the worker is right-handed, the left hand is set as the worker's control hand signal. It may be a hand setting for extraction.

Therefore, if the worker's main hand when performing a task is the right hand, the worker can perform the task and control work instruction information at the same time by making gestures with the left hand, and vice versa, if the left hand is the main working hand. It is also possible.

Meanwhile, the edge server 100 is the main server 300 and can provide at least one of image data, the number of occurrences of a task error, the number of gesture inputs, the task yield, and the task time for a plurality of tasks.

The main server 300 may automatically generate an analysis report for each worker who performed each task using the data received from the plurality of edge servers 100 and provide the data to the administrator terminal 400.

Through this, the main server can calculate summed data about a plurality of edge servers 100 and a plurality of workers working on each edge server 100, and this summed data includes total production, total production yield, and total work. It may also be included in the analysis report as an indicator such as time.

Referring to FIG. 6, the procedure for performing a work guidance method using the work guide device 200 according to an embodiment of the present invention will be described.

First, the work guide device 200 mounted on the worker's head captures the worker's front view and transmits the captured image to the edge server 100, collects the fastening state data through the fastening device, and sends the collected fastening state data to the edge server 100. Transmit to the edge server 100 (S101).

The edge server 100 analyzes the image received from the work guide device 200 through a previously learned task analysis model to determine the type of work the worker is working on and whether there is a work error (S102).

The edge server 100 provides work instruction information corresponding to the identified work type and whether or not there is a work error through the work guide device 200 (S103).

One embodiment of the present invention may also be implemented in the form of a recording medium containing instructions executable by a computer, such as program modules executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include all computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described with respect to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as unitary may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

100: Edge server 200: Operation guide device
210: support part 220: main body part
230: leg part 231: camera device
300: Main server 400: Administrator terminal

Claims (13)

In a work guiding method using a work guiding device performed by a system,
(a) A work guide device mounted on the worker's head photographs the worker's front view and transmits the captured image to the edge server, collects fastening state data through a fastening device, and sends the collected fastening state data to the edge server. Transferring to;
(b) the edge server analyzing the image received from the work guide device through a previously learned work analysis model to determine the type of work the worker is working on and whether there is a work error; and
(c) providing, through the work guide device, work instruction information corresponding to the identified work type and whether or not there is a work error, by the edge server; Including,
The work instruction information is at least one of guidance information for the current work, information about errors, and guidance information for the next work,
The work analysis model is a model that is learned using the operator's motion images for each type of work and images of work objects and work tools as learning data, and outputs the type of work and the presence of work errors from the input images. A work guide device. How to guide your work through. According to paragraph 1,
In step (a),
While the fastening device is in contact with the fastening screw, it rotates according to a control signal to rotate the fastening screw to fasten the fastening screw to the work object, and the number of rotations of the motor of the fastening device and the fastening screw per unit time generated during the fastening operation. (RPM), a work guide method using a work guide device, wherein at least one of the torque values is collected as fastening state data and provided to the edge server. According to paragraph 1,
In step (b),
(b-1) classifying the image received from the work guide device by the edge server into a work object, work tool, worker's body, and other objects according to an image analysis technique;
(b-2) The images of the classified work objects and work tools are compared with the previously saved images for each work type and work progress, and if they match more than a preset value, the current work progress is judged as the above work progress, and the classified Comparing the image of the work object and the fastening device with a pre-stored image for each work error, and if the image matches a preset value or more, determining whether there is a work error in the current work; and
(b-3) If the images of the classified work and work tools match the pre-stored work progress image and a pre-set value, determining that the work progress is the pre-stored work progress includes the step of determining , A method of guiding work through a work guiding device. According to paragraph 3,
In step (b-2),
Based on the RPM and torque values of the fastening state data collected from the fastening device, abnormal fastening of the fastening screw and the work object is detected by comparing it with a previously stored graph for each type of abnormal fastening to determine whether there is a work error, A method of guiding work through a work guiding device. According to paragraph 1,
The work guide device is,
A support portion that rests on the bridge of the worker's nose and supports the main body;
The main body portion has a predetermined thickness and is formed in a semicircular shape at a predetermined distance above the worker's eyebrows, and is connected to an upper side of the support portion;
a leg portion that extends from the main body portion and is fixed to the worker's ear, and includes a camera device attached to the outer side of the worker's head to capture the worker's front view; and, from the edge server inside the main body portion, , A work guide method using a work guide device, which includes a built-in bone conduction speaker device that outputs a voice corresponding to the work instruction information and a control signal voice according to the received control signal. According to clause 5,
The work guide device is,
It is formed at a predetermined distance from the operator's eyeballs so as not to block the captured image of the camera device, and extends from the bottom of the main body, and further includes a glass part made of a transparent or translucent material so as not to block the operator's field of view. A work guide method using a work guide device. According to paragraph 1,
The work guide device is,
A main body portion that has a predetermined thickness and is formed in a semicircular shape and supported in contact with the back of the worker's head;
a support portion that has a predetermined thickness, is in contact with the worker's temporal portion, extends from the main body portion above the worker's ear to the worker's temple, and rests on the worker's ear to support the main body portion;
A camera device attached to the support part in an outer direction of the worker's head to capture the front view of the worker,
A work guide method using a work guide device, wherein the main body has a built-in bone conduction speaker device that outputs a voice corresponding to work instruction information and a control reception signal voice according to a control signal received from an edge server. According to paragraph 1,
In step (c),
The work guide device,
Providing work instruction information corresponding to the type of work, work progress, and work error received from the edge server in the form of a voice and a control signal corresponding to the work instruction information through a bone conduction speaker device inside the work guide device. In, a method of guiding work through a work guiding device. According to clause 8,
In step (c),
When it is determined that the order of the current work is incorrect or a work error has occurred, a work guide method using a work guide device allows the work guide device to provide a work error signal including at least one of sound, vibration, and image. . According to clause 6,
In step (c),
The work guide device,
A work guide through a work guide device, which provides work instruction information corresponding to the type of work and work progress received from the edge server through an AR image displayed on the glass portion of the work guide device overlapping with the worker's field of view. method. According to paragraph 1,
In step (c),
If the worker determines the type of work being done and whether there is a work error, and determines that the work has been completed without error, the preset next work instruction information is provided to the work guide device, and the information received from the camera device of the work guide device is provided. When the worker's hand part in the image is recognized and a control hand signal is extracted from the recognized hand part, work instruction information corresponding to the extracted control hand signal is provided to the work guide device in priority over the preset next work instruction information. A work guide method using a work guide device. According to paragraph 1,
(d) The edge server provides at least one of image data, number of task errors, number of gesture inputs, task yield, and task time for a plurality of tasks to the main server, and the main server performs each task. A work guide method using a work guide device that further includes the step of automatically generating an analysis report for each worker and providing it to an administrator terminal. In the work guide system through the work guide window,
A head-mounted bone conduction work guide device that captures the worker's front view and transmits the captured images to the server;
A fastening device that collects fastening state data and transmits the collected fastening state data to an edge server; and
The images and fastening state data received from the work guide device and the fastening device are analyzed through a previously learned work analysis model to determine the type of work the worker is working on and whether there are work errors, and the identified work type and work progress. Including; an edge server that provides at least one of work instruction information about work instructions and errors and guidance information about the next work through the work guide device;
The work instruction information is at least one of information about errors in the current work and guidance information for the next work,
The work analysis model is a model that is learned using the operator's motion images for each type of work and images of work objects and work tools as learning data, and outputs the type of work and the presence of work errors from the input images. A work guide device. Work guide system through.

Images