WO2021006183A1

WO2021006183A1 - Task classification system and task classification program

Info

Publication number: WO2021006183A1
Application number: PCT/JP2020/026072
Authority: WO
Inventors: 巨樹松山
Original assignee: コニカミノルタ株式会社
Priority date: 2019-07-10
Filing date: 2020-07-02
Publication date: 2021-01-14
Also published as: JPWO2021006183A1; JP7347509B2

Abstract

Provided is a task classification system capable of classifying tasks into those with facilities in operation and those with facilities under suspension.　A task classification system (1) comprising a facility operating status acquisition unit (12) that acquires the operating status of a facility (50), a camera (20) that captures a range including a worker (30), a worker position/orientation determination unit (14) that determines the position and orientation of the worker (30) from video data captured by the camera (20), and a task classification unit (15) that classifies a changeover task according to a combination of the operating status of the facility (50) and the position and orientation of the worker (30).

Description

Work classification system and work classification program

The present invention relates to a work classification system and a work classification program.

At the production site, there is an increasing need for high-mix low-volume production for mass customization. In high-mix low-volume production, many setup changes are required. The setup change work causes a work loss. For this reason, in high-mix low-volume production, how to suppress work loss in setup change work is an issue.

In order to reduce work loss, it is first necessary to classify the work as to what kind of work is being performed at the production site.

As the conventional technology, for example, there are the following technologies.

In Patent Document 1, the posture or motion of the worker is determined, and the work motion directly related to the work and the non-work motion not directly related to the work are extracted from the determined posture or motion of the worker. Then, in this technology, the burden value for the extracted operation is calculated, and the load evaluation information for the non-work operation is presented using the result, and this information is used for equipment layout, line design, line improvement, and safety improvement. It is used for such purposes.

Further, in Patent Document 2, the alarm information indicating that the worker is performing non-routine work and the work record information of the worker are acquired, and the alarm information and the work record information are associated with each other to perform the work capable of routine work. Work is allocated to each worker according to the number of people. As a result, in this technology, work management is performed by distinguishing between non-routine work and routine work.

Further, in Patent Document 3, a first worker image in which a worker at the start of work is shown and a second worker image in which a worker at the end of work is shown are acquired, and work related to one work item is obtained. Measure the work time required for. Further, in this technique, the work matter is recognized from the second worker image, and the information on the worker and the implementation status for each work matter is displayed based on the measurement result of the work time and the work recognition result. As a result, in this technique, the worker is identified from the image, the working time of the worker is acquired, and the work matter is recognized. Therefore, in this technique, it is not necessary for the operator to input information such as operating the terminal, so that it is possible to avoid a decrease in work efficiency.

JP-A-2017-68431 JP 2011-216014 JP-A-2015-225630

However, in the technique of Patent Document 1, the work is only classified into the work operation directly related to the work and the non-work operation not related to the work. In this case, it is not possible to classify work according to the operating state of the equipment, such as work that requires the equipment to be stopped and work that can be performed without stopping the equipment, such as setup change work.

In the technique of Patent Document 2, recognition of the occurrence of non-routine work is alarm information issued by an operator, and work classification such as non-routine work and routine work is not automated. Further, even in the technique of Patent Document 2, classification is not performed according to the operating state of the equipment.

In the technique of Patent Document 3, the start / end of work is determined by the image at the time of entering / exiting the work area, and the work can be classified only by the unit of entering / exiting the work area. Therefore, even in the technique of Patent Document 3, classification is not performed according to the operating state of the equipment.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a work classification system and a work classification program capable of classifying work in which the equipment is in operation and work in which the equipment is stopped. Is.

The above object of the present invention is achieved by the following means.

(1) The equipment operation status acquisition unit that acquires the operation status of the equipment,
A camera that captures the area including the worker,
A worker position / orientation determination unit that determines the position and orientation of the worker from the video data captured by the camera, and
A work classification unit that classifies setup change work based on the combination of the operating state of the equipment and the position and orientation of the worker.
Has a work classification system.

(2) The setup change work is a work when one product is produced or processed.
A predetermined range within the shooting range of the camera and in the vicinity of the equipment is set as the first area.
The work classification unit classifies the setup change work as an internal setup work when the worker is present in the first area and faces the equipment while the equipment is stopped, and the equipment is in operation. When the worker is not facing the equipment in the first area, it is classified as an external setup work, and the worker is not facing the equipment in the first area while the equipment is stopped. The work classification system according to (1) above, which classifies as waste when it is outside the first area, and classifies it as absent when the worker is outside the first area while the equipment is in operation.

(3) The setup change work is a work when one product is produced or processed.
A predetermined range within the shooting range of the camera and in the vicinity of the equipment is defined as a first region, and a range outside the first region is defined as a second region.
The work classification unit classifies the setup change work as an internal setup work when the worker is present in the first area and faces the equipment while the equipment is stopped, and the equipment is in operation. When the worker is not facing the equipment in the first area and when the worker is present in the second area, it is classified as an external setup work, and the worker is in the first area while the equipment is stopped. If the equipment is not facing the facility or is outside the first area, it is classified as waste, and if the worker is outside the first area and the second area while the facility is in operation, it is regarded as absent. The work classification system according to (1) above, which classifies.

(4) Further having a worker identification unit for identifying the worker,
The work classification system according to any one of (1) to (3) above, wherein the work classification unit associates the classified classification results with each of the workers identified by the worker identification unit.

(5) The worker position / orientation determination unit extracts the coordinate values of a predetermined joint or skeleton of the worker from the video data, and determines the position and orientation of the worker from the coordinate values. The work classification system according to any one of 1) to (4).

(6) A video recording unit that records the video data and
The work classification system according to any one of (1) to (5) above, further comprising a classification result classified by the work classification unit and a linking unit for linking the video data.

(7) The linking unit further adds the operating state acquired by the equipment operating state acquisition unit and the position and orientation of the worker determined by the worker position / orientation determination unit to the classification result and the video data. The work classification system according to (6) above, which is linked.

(8) The stage (a) of acquiring the operating status of the equipment and
The step (b) of determining the position and orientation of the worker from the video data in which the range including the worker is captured, and
The stage (c) of classifying the setup change work based on the combination of the operating state of the equipment and the position and orientation of the worker, and
A work classification program that allows a computer to execute.

(9) The setup change work is a work when one product is produced or processed.
In the step (c), a predetermined range in the vicinity of the equipment is set as the first region within the range in which the video data is captured, and the setup change work is performed by the worker in the first region while the equipment is stopped. It is classified as an inner setup work when it exists inside and faces the equipment, and when the worker is not facing the equipment in the first area while the equipment is in operation, the outer setup work. When the equipment is stopped and the worker is not facing the equipment in the first area or outside the first area, it is classified as waste, and the worker is operating the equipment. The work classification program according to (8) above, which classifies as absent when it is outside the first area.

(10) The setup change work is a work when one product is produced or processed.
In the step (c), the setup change operation is performed by setting a predetermined range in the vicinity of the equipment as a first region and a range outside the first region as a second region within the range in which the video data is captured. Is classified as an internal setup work when the worker is present in the first area while the equipment is stopped and is facing the equipment, and the worker is in the first area while the equipment is in operation. When the equipment is not facing the equipment and when it is present in the second area, it is classified as an external setup work, and the worker is not facing the equipment in the first area while the equipment is stopped. The case or when it is outside the first area, it is classified as waste, and when the worker is outside the first area and the second area, it is classified as absent, according to the above (8). Work classification program.

(11) The step (d) of identifying the worker and
The work classification according to any one of (8) to (10) above, further comprising a step (e) of associating the classification results classified according to the step (c) with each identified worker. program.

(12) In the step (b), the coordinate values of a predetermined joint or skeleton of the worker are extracted from the video data, and the position and orientation of the worker are determined from the coordinate values. The work classification program according to any one of (11).

(13) The step (f) of recording the video data and
The work classification program according to any one of (8) to (12) above, further comprising a classification result classified by the step (c) and a step (g) for associating the video data.

(14) The work classification program according to (13) above, wherein the step (g) further links the operating state, the position and orientation of the worker, to the classification result and the video data.

According to the present invention, the operating state of the equipment is acquired, the position and orientation of the worker are determined, and the setup change work is classified from the combination thereof. Thereby, in the present invention, the work in which the equipment is in operation and the work in which the equipment is stopped can be classified.

It is explanatory drawing for demonstrating the functional structure of the work classification system of embodiment. It is explanatory drawing for demonstrating the position and direction of a worker with respect to equipment. It is explanatory drawing for demonstrating the position and direction of a worker with respect to equipment. It is a classification judgment table for classifying work. It is a time chart which displayed each information and classification result. It is a pie chart showing the classification result of work by the time ratio. It is a table which shows the classification result of work together with time and work time. It is a figure which shows the reproduced video example. It is a table which shows the work contents, the acquired information, and the classification result in detail. It is a flowchart which shows the processing procedure of work classification.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted. In addition, the dimensional ratios in the drawings are exaggerated for convenience of explanation and may differ from the actual ratios.

FIG. 1 is an explanatory diagram for explaining the functional configuration of the work classification system of the embodiment.

The work classification system 1 has a server 10, a camera 20 connected to the server 10, and a terminal 18 connected to the server 10.

The server 10 has the functions of the worker identification unit 11, the equipment operation status acquisition unit 12, the video recording unit 13, the worker position / orientation determination unit 14, the work classification unit 15, the product information acquisition unit 16, and the linking unit. Has 17.

The server 10 is a computer and has a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), and an interface (IF) for external connection. Further, the server 10 has an input device such as a keyboard and a mouse, and an output device such as a display.

In the server 10, each function is realized by the CPU reading and executing the program stored in the ROM or HDD.

Further, in the server 10, the interface communicates with the equipment 50, the camera 20, and the terminal 18. The interface includes, for example, a network interface according to a standard such as Ethernet (registered trademark), SATA, PCI Express, USB, and IEEE 1394, and various local connection interfaces such as a wireless communication interface such as Bluetooth (registered trademark) and IEEE 802.11. The interface may also be a line for making a local analog connection with the equipment 50 and the camera 20. Further, the interface may be a wireless LAN (WiFi) or a mobile phone line for connecting to the terminal 18.

The terminal 18 is a computer. As the terminal 18, for example, a personal computer (PC (Personal Computer)), a tablet, a smartphone, or the like is used. These are computers used by workers, managers, and other stakeholders. These computers have input and output devices such as keyboards, mice, touch panels, and displays.

Each function of the server 10 will be explained.

The worker identification unit 11 individually recognizes the worker 30 that has entered the work area (details will be described later). The worker 30 is identified, for example, by using an RFID (Radio Frequency Identification) tag 31 or by using biometric authentication such as face authentication, iris authentication, and fingerprint authentication. For example, when the RFID tag 31 is used, the RFID tag 31 is carried by the worker 30 or attached to a work clothes, a hat, a helmet, or the like. On the other hand, a reading device for reading the RFID tag 31 is installed in the work area. Then, the worker identification unit 11 causes the RFID tag 31 to be read by the reading device to acquire the identification information of each worker. Similarly, when iris recognition is used, the worker identification unit 11 identifies the worker 30 by using these authentication sensors and cameras.

The equipment operating state acquisition unit 12 acquires the operating state of the equipment 50. The operating state of the equipment 50 is, for example, an operation by normal operation for production, a completion stop when machining is completed and stopped, an abnormal stop when an abnormality occurs, and a stop where the equipment 50 is not used.

The equipment operating state acquisition unit 12 acquires the operating state of the equipment by, for example, detecting the emission color of the laminated signal lamp 51 provided in the equipment 50 or detecting the switching signal. When detecting the emission color or switching signal, a dedicated sensor (not shown) for detecting them is used. The equipment operating state acquisition unit 12 (server 10) is connected to a sensor that detects the emission color and the switching signal.

The laminated signal lamp 51 is also called a three-color laminated lamp, a signal tower, or the like, and most of them emit light of three color lamps of green, yellow, and red according to the operating state of the equipment 50. The laminated signal light 51 is usually green during normal operation, yellow when stopped due to production completion, and red when an abnormality occurs.

Further, when the equipment operating state acquisition unit 12 outputs a signal indicating the operating state from the equipment 50, the equipment operating state acquisition unit 12 may acquire the signal. In this case, the equipment operating state acquisition unit 12 (server 10) is connected to the equipment 50.

Further, the equipment operating state acquisition unit 12 may acquire the operating state from the video data of the camera 20. In this case, the laminated signal light 51 is placed in the shooting range of the camera 20. The equipment operating state acquisition unit 12 detects the emission color of the laminated signal lamp 51 from the video data.

The video recording unit 13 records video data from the camera 20. The video recording unit 13 is, for example, an HDD in the server 10. Video data is recorded over time.

The worker position / orientation determination unit 14 detects the position and orientation of the worker 30 from the video data.

The camera 20 has a shooting range 25 including a work area in which the worker 30 works on the equipment 50. As described above, when the operating state of the equipment 50 is detected from the video data of the camera 20, the laminated signal light 51 is included in the shooting range 25.

Existing technology can be used to detect the position and orientation of the worker 30. For example, open pose (open pose (see https://github.com/CMU-Perceptual-Computing-Lab/openpose, etc.)), deep pose (deep pose (https://www.slideshare.net/mitmul/deeppose-) Use technology that recognizes the posture by estimating the human skeleton from dual video data (video) such as human-pose-estimation-via-deep-neural-networks)) (hereinafter referred to as skeleton recognition technology). Can be done. In addition, as the skeleton recognition technology, a technology using a depth camera (RBG-D camera) or TOF (Time of Flight), such as Kinect (registered trademark) of Microsoft Corporation, can be used.

The skeleton recognition technology can recognize the posture of a person from two-dimensional video data. Therefore, a general movie camera can be used as the camera 20, and the system configuration is simplified.

2a and 2b are explanatory views for explaining the position and orientation of the worker 30 with respect to the equipment 50. FIG. 2a is an example in which the equipment 50 is installed independently, such as an automatic processing machine 50a. FIG. 2b shows an example in which the equipment 50 is a production line 50b for assembly line work.

In detecting the position of the worker 30 using the skeleton recognition technique, the first region 41 and the second region 42 are set in the photographing range 25 as shown in FIGS. 2a and 2b. The first area 41 is a predetermined range in the vicinity of the equipment, for example, a range in which the worker 30 can directly work on the equipment 50 or the product set in the equipment 50. Specifically, the first region 41 is, for example, a range in which the worker 30 can reach the automatic processing machine 50a, the production line 50b, and the products set in them if the worker 30 reaches out. On the other hand, the second region 42 is a region outside the first region 41. The first region 41 and the second region 42 are exclusively set.

The shooting range 25 is a range including at least the first area 41 and the second area 42. In FIGS. 2a and 2b, the outside of the second region 42 is also photographed.

The worker position / orientation determination unit 14 determines from the video data whether the worker 30 exists in the first area 41, exists in the second area 42, or does not exist (absence) in both areas.

In the skeleton recognition technology, the person in the image is specified by the three-dimensional coordinate value. The three-dimensional coordinate system to be used and the coordinate origin are set in advance. Further, coordinate values indicating the ranges of the first region 41 and the second region 42 are also set in advance in the three-dimensional coordinate system.

For example, as shown in FIG. 2a, when the equipment 50 is an automatic processing machine 50a, work is performed on the outer surface of the automatic processing machine 50a, for example, an operation panel, an outer door, a raw material supply port, an access panel, etc., as the first area 41. The range is set so that the person 30 can access it (the range shown in the figure a). The second region 42 is set further outside from the outer peripheral end of the first region 41 (range b in the figure).

Further, for example, as shown in FIG. 2b, when the equipment 50 is the production line 50b, a range of a certain distance is set as the first region 41 in parallel with the production line 50b (range in the figure a). The range of a certain distance is a range in which the worker 30 can access the work on the production line. The second region 42 is set further outside from the outer peripheral end of the first region 41 (range b in the figure).

In any of the above cases, the second area 42 is preferably a range in which the worker 30 can immediately move to work, for example. Specifically, the second region 42 is, for example, a range of 1 m from the outer peripheral end of the first region 41.

Of course, the ranges of the first region 41 and the second region 42 are merely examples and can be set arbitrarily.

The worker position / orientation determination unit 14 first detects the position of the worker 30. The worker position / orientation determination unit 14 extracts the coordinate values of predetermined joints from the skeleton of the worker 30 obtained by using the skeleton recognition technique, and determines in which region the worker 30 exists. Or determine if it does not exist in both. The predetermined joints are, for example, the neck, shoulders, elbows, wrists, hips, knees, ankles, and the like, and may be determined in advance. When the worker 30 is present so as to straddle the boundary between the first region 41 and the second region 42, it is assumed that the worker 30 is present in the region including the predetermined joint of the worker 30, for example. To do. As a result, it is detected whether the worker 30 is present in either region or is not present in either region. In addition, the coordinate value of the skeleton provided by the skeleton recognition technique may be used instead of the coordinate value of the joint.

The worker position / orientation determination unit 14 then detects the orientation of the worker 30. The detection of the orientation of the worker 30 is executed only for the worker 30 existing in the first region 41.

The worker position / orientation determination unit 14 determines that the direction in which the arm (particularly the wrist joint) protrudes with respect to the joint position of the neck or shoulder is the direction in which the person is facing. In FIGS. 2a and 2b, the worker 30a shown by the solid line is determined to be facing the equipment 50 because the direction in which both arms are protruding is the direction of the equipment 50. On the other hand, the worker 30b shown by the dotted line is determined not to face the equipment 50 because both arms are facing in a direction other than the equipment 50 (diagonally backward in the figure).

In the present embodiment, if at least one arm protrudes in the direction of the equipment 50 with respect to the neck or shoulder, it is determined that the worker 30 is facing the equipment 50. During the work on the equipment 50, both arms of the worker 30 may protrude in the direction of the equipment 50, or only one hand may protrude in the direction of the equipment 50. For example, work on the operation panel such as changing the setting of the machining program or restarting can be performed with one hand. Therefore, in the present embodiment, when at least one arm is facing the equipment 50, it is determined that the worker 30 is facing the equipment 50.

When a plurality of workers 30 are within the photographing range 25, the worker position / orientation determination unit 14 detects the position of each worker 30, and further, the plurality of workers 30 are present in the first area 41. If so, the orientation of each worker 30 is determined.

A two-dimensional coordinate system may be used to determine the position and orientation of the worker. When the two-dimensional coordinate system is used, for example, the camera 20 is installed at a position where the camera 20 is photographed from above the worker 30 (may be from diagonally above). A two-dimensional coordinate system is set in the video data, and a first region 41 and a second region 42 are set. Then, the worker position / orientation determination unit 14 determines whether or not the worker 30 exists in each area in the same manner as described above.

The work classification unit 15 converts the information on the operating state of the equipment 50 from the equipment operating state acquisition unit 12 and the information on the state (position and orientation) of the worker 30 from the worker position / orientation determination unit 14 into the equipment 50. Classify the work being done on it.

FIG. 3 is a classification judgment table for classifying work.

First, the work classification unit 15 classifies the state of the worker 30 from the position and orientation of the worker 30 into three categories: anti-equipment work, non-equipment work, and absence.

The equipment work is a case where the worker 30 existing in the first area 41 is facing the equipment 50.

The non-equipment work is a case where the worker 30 existing in the first area 41 does not face the equipment 50 and a case where the worker 30 exists in the second area 42. In this non-equipment work, there is a worker 30 in the vicinity of the equipment 50, and the work on the equipment 50 is not performed, but the work on the equipment 50 can be performed immediately. If it is possible to work on such equipment 50, it is classified as different from absence.

Absence is when the worker 30 does not exist in either the first area 41 or the second area 42.

Subsequently, the work classification unit 15 classifies the work from the combination of the operation state of the equipment 50 from the equipment operation state acquisition unit 12 and the work state of the worker 30.

The work to be classified is the setup change work. The setup change work is a work that is performed after the production or processing of a certain product is completed, until the jig or tool of the equipment 50 is changed in order to produce or process the next product, and the production or processing of the product is completed. Is. Jig tool changes include changing jigs and tools, changing equipment programs required for production and machining, and replacing and installing additional parts required for products to be produced or machined.

This setup change work is classified into the following four categories: inner setup work, outer setup work, waste, and absence.

The internal setup work is work performed on the equipment 50 while the equipment 50 is stopped. The internal setup work includes, for example, replacement of jigs and tools and tools, change of equipment program, installation of additional parts, setting of pre-processed product to equipment 50, and the like. These operations are performed while the equipment 50 is stopped. Therefore, when the equipment is stopped (yellow, red, and when the lights are off), the combination of equipment work is classified as internal setup work.

The external setup work is a work performed while the equipment 50 is in operation. The external setup work is performed after preparatory work such as preparing jigs and tools and additional parts for the production or processing of the next product, carrying out the product whose production has been completed or processed, and cleaning and cleaning around the equipment 50. Cleaning up work, etc. It is efficient that these operations are performed while the equipment 50 is in operation, i.e., while the previous product is being produced or processed. Therefore, when the equipment is in operation (when it is green) and the combination of non-equipment work is performed, it is classified as external setup work.

Waste is a state in which work is being performed on other than the equipment 50 even though the equipment 50 is stopped. Waste is, for example, finding the jigs and tools needed for the next product to be produced or processed, or going to pick up additional parts. Therefore, non-equipment work when the equipment is stopped in yellow, red, and off is classified as waste.

Absence is when the worker 30 does not exist in either the first area 41 or the second area 42 while the equipment is in operation. During the operation of the equipment, the worker 30 does not usually have to be in the vicinity of the equipment 50, so that it is in a normal state and is simply classified as absent.

In addition, the work classification unit 15 records the start and end times of each classified work together. The start and end of each classified work starts at the classified time and ends at the time when the next classification is switched to. Specifically, it is recorded at the following timing. When the worker 30 is in the first area 41, the time when the direction is changed is recorded. When the worker 30 is present in the second area 42, the time when the worker 30 enters the first area 41 from the second area 42 or the time when the worker 30 goes out of the second area 42 is recorded. When the worker 30 is absent, the time when the worker 30 enters the second area 42 is recorded. If there is no change in the position and orientation of the worker 30, the time when the operating state of the equipment 50 has changed is recorded.

Further, the work classification unit 15 records the classification results in association with each worker 30 identified by the worker identification unit 11. The classification result associated with each worker 30 is recorded in, for example, the HDD of the server 10.

The product information acquisition unit 16 acquires product information, which is information on products produced and processed by the equipment 50. The product information is acquired, for example, when the product before processing arrives at the equipment 50, and is recorded together with the time at that time. The time of the product information is recorded when the processing is completed. The product information may be read from, for example, a barcode attached to the product, an RFID tag 31, or the like. Further, the product information may be input by the worker 30 from the equipment 50 or a terminal installed near the equipment 50. Further, the product information may be recalled in order from which the server 10 or another computer stores the product list.

The linking unit 17 includes worker information from the worker identification unit 11, operating status of the equipment 50 from the equipment operating status acquisition unit 12 (referred to as equipment operation information), product information from the product information acquisition unit 16, and a work classification unit. The work classification results from 15 and the video data recorded in the video recording unit 13 are linked (linked). The linking unit 17 links each information, the classification result, and the video data on the basis of time.

FIG. 4 is a time chart displaying each information and classification result. In the figure, "operation" means that the equipment 50 is in operation, "pair" means work with equipment, "non" means non-equipment work, "inside" means internal setup work, and "outside" means external setup. Each work is shown. The same applies to other figures.

As shown in FIG. 4, product information, equipment operating status, working status, and classification results are shown as a time chart, which makes it easier to understand how these were performed over time.

FIG. 5 is a pie chart showing the work classification results in terms of time.

As shown in FIG. 5, by showing the classification result as a pie chart of the time ratio, it becomes easy to understand how much time ratio each classified work is performed.

FIG. 6 is a table showing the work classification results together with the time and work time.

As shown in FIG. 6, by showing the work classification result together with the classified time and work time, it becomes easy to understand the start and end times of each work and the work time required for the work. In addition, this table also displays playback icons for playing back video data. The playback icon is linked with the video data recorded in the video recording unit 13 based on the time.

FIG. 7 is a diagram showing an example of the reproduced video. For example, when the playback icon indicated by the dotted circle 300 in the figure is clicked, the image shown in FIG. 7 is played back. In this image, the state in which the worker 30 is working on the automatic processing machine 50a is projected.

Note that the playback of video data is not limited to playback from a table with a playback icon as shown in FIG. For example, in the time chart shown in FIG. 4, the video data may be reproduced by clicking each information or the classification result portion.

The video data to be played is, for example, from the start to the end of the specified work classification. Further, the video data to be reproduced may be reproduced including before and after from the start to the end of the designated work classification.

Further, the video data to be reproduced may be reproduced from an arbitrary time regardless of the work classification. When the video data is reproduced from an arbitrary time regardless of the work classification, it is preferable to display the classification result according to the reproduced video data. If the video data being played straddles the classification result, the display of the classification result is switched.

FIG. 8 is a table showing the work contents, acquired information, and classification results in detail.

As shown in FIG. 8, by showing the work contents, the acquired information, and the classification result in detail, it becomes possible to understand how each collected information is classified. In addition, in this table, since the work contents are linked to each information and the classification result, it becomes possible to know what kind of work was concretely performed. Further, since the workers 30 who are performing the work are also individually displayed in this table, it is possible to know how much time each worker 30 took for which work.

Such detailed information will be useful when making improvement plans and production plans for shortening takt time. Also in the table shown in FIG. 8, each information or classification result may be associated with video data, and the video data may be reproduced by clicking each information or classification result in the table.

In the display examples shown in FIGS. 4 to 8, each information linked by the linking unit 17 is output on a time basis or in a time ratio. These outputs are performed in response to a request from the terminal 18, for example, and are displayed on the display of the terminal 18.

Next, the processing procedure for classifying work will be described. FIG. 9 is a flowchart showing a processing procedure of work classification. By executing the work classification program created based on this processing procedure, the server 10 executes the functions of the respective parts already described.

First, the server 10 identifies the worker 30 (S1).

Subsequently, the server 10 acquires the operating state of the equipment 50 (S2).

Subsequently, the server 10 acquires video data from the camera 20 (S3).

Subsequently, the server 10 acquires the product information (S4).

Subsequently, the server 10 detects the position and orientation of the worker 30 from the video data (S5).

Subsequently, the server 10 classifies the work from each obtained information (S6).

Subsequently, the server 10 associates the obtained information, the classification result, and the video data (S7).

Subsequently, the server 10 ends this process if there is an input to end the process (S8: YES). On the other hand, if there is no input for the end of processing (S8: NO), the server 10 returns to S1 and continues this processing.

In the above processing procedure, if the processing order of S1 to S6 is such that S3 is processed before S5 and S1 to 5 are processed before S6, each step of S1 to 4 is performed. It may be in any order.

According to the present embodiment described above, the following effects are obtained.

In this embodiment, the operating state of the equipment 50 and the working state of the worker 30 are acquired, and the setup change work is classified into four, an inner setup work, an outer setup work, waste, and absent, according to the combination thereof. .. Thereby, in the present embodiment, the work in which the equipment 50 is in operation, the work in which the equipment 50 is stopped, and the work unrelated to the equipment 50 can be classified. Therefore, in the present embodiment, what was conventionally classified manually by a person can be automatically classified, and the classification work can be greatly streamlined.

Further, in the present embodiment, it is decided that each acquired information, classification result, and video data are synchronized and linked according to time. Thereby, in the present embodiment, each associated information and the classification result can be displayed as a time chart, a graph, a table, or the like. Further, since the video data related to them is also linked, the video data related to each information and the classification result can be easily reproduced. In particular, in the present embodiment, by associating the video data, the work situation can be visualized and the improvement analysis can be made more efficient. Further, in the present embodiment, from the analysis result, the setup change work can be shortened, the downtime of the equipment 50 can be reduced, and the productivity can be improved.

Although the embodiment of the present invention has been described above, various modifications are possible. Although the embodiment has been described as a function of the server 10, the function of the server 10 can be performed by a small computer. For example, each function described as an embodiment may be executed in the camera 20 by incorporating it into the camera 20 such as a board PC or integrating it with the camera 20. By doing so, this embodiment can be implemented only by installing the camera 20.

Further, as the server 10, a cloud server on the Internet may be used. When using a cloud server, the equipment 50 and the camera 20 are provided with an interface capable of connecting to the Internet, and the operating state of the equipment 50 and the video data of the camera 20 are transmitted to the cloud server. The connection between the equipment 50 and the camera 20 and the Internet may be a wireless connection or a wired connection. When using a cloud server, the initial investment can be suppressed because the existing webcam that can connect to the Internet can be used for the camera 20 simply by attaching an interface for connecting to the Internet to the equipment 50. In addition, when using a cloud server, if the operating state of the equipment 50 is acquired from the emission color of the laminated signal light 51 in the video data, only the installation of the webcam is required, so the initial investment can be further reduced. Can be done.

Further, not only the skeleton recognition technique but also other sensors may be used to detect the position and orientation of the worker 30. For example, it is possible to use a technique of estimating a person's posture by obtaining a distance value to each part of a person by a rider (LIDAR (Light Detection and Ringing)) that obtains a distance value to an object. In addition, a technique of determining the direction of a person from the temperature of the person's face using an infrared sensor (thermal camera) can be used.

Further, in the embodiment, the worker identification unit 11 recognizes the worker 30 individually, but if it is not necessary to associate the work classification result for each worker 30, the individual worker 30 is recognized. No need to identify. If the individual worker 30 is not identified, the worker identification unit 11 may be omitted. If the worker identification unit 11 is not provided, the RFID tag 31, its reading device, the device for biometric authentication, and the like are naturally unnecessary, so that the capital investment cost can be reduced.

Further, the operating state of the equipment 50 may be acquired by sound when the equipment 50 is equipped with an acoustic alarm that notifies the operation or abnormal stop by sound.

Further, the laminated signal lamp 51 is not limited to three colors, and can be applied to a two-color laminated signal lamp 51, a one-color laminated signal lamp 51, and the like. For example, in the case of the two-color laminated signal lamp 51, the normal operation of green and the abnormal stop of red are performed. In this case, since green is in operation and red and off are stopped, processing may be performed accordingly. Similarly, the same applies to the case of one color and the case where a laminated signal lamp 51 having four or five colors is used.

Further, in the embodiment, the second region 42 is provided. In the embodiment, by providing the second area 42, it can be seen that the worker 30 is not working on the equipment 50 but is in a position where he can work immediately. However, as an embodiment, the work classification can be carried out even if only the first region 41 is used. When only the first area 41 is set, the case where the worker 30 existing in the first area 41 does not face the equipment 50 is regarded as non-equipment work, and the case where the worker 30 does not exist in the first area 41 is absent. And. Therefore, the work classification in this case is classified as an external setup work when the worker 30 is present in the first area 41 and does not face the equipment 50 while the equipment is in operation. Further, when the equipment is in operation and the worker 30 does not exist in the first area 41, it is classified as absent. Other classifications are the same as those shown in FIG.

In addition, the conditions and numerical values used in the description of the embodiment are for explanation purposes only, and the present invention is not limited to these conditions and numerical values.

The work classification program according to the present invention can also be realized by a dedicated hardware circuit. In addition, this work classification program is provided by a computer-readable recording medium such as a USB (Universal Serial Bus) memory or a DVD (Digital Versaille Disc) -ROM (Read Only Memory), or is provided by a computer-readable recording medium such as the Internet regardless of the recording medium. It is also possible to provide it online via the network of. In this case, this work classification program is usually stored in a magnetic disk device or the like that constitutes a storage unit. In addition, this work classification program can be provided as a single application software, or can be provided by being incorporated into another software as one function.

Further, the present invention can be modified in various ways based on the configurations described in the claims, and these are also within the scope of the present invention.

This application is based on Japanese Patent Application No. 2019-128731 filed on July 10, 2019, the disclosure of which is incorporated as a whole by reference.

1 Work classification system,
10 servers,
11 Worker identification unit,
12 Equipment operation status acquisition department,
13 Video recording section,
14 Worker position / orientation determination unit,
15 Work classification department,
16 Product Information Acquisition Department,
17 Tying part,
18 terminals,
20 cameras,
25 shooting range,
30, 30a, 30b workers,
41 First area,
42 Second area,
50 equipment,
50a automatic processing machine,
50b production line,
51 Stacked signal light.

Claims

The equipment operating status acquisition unit that acquires the operating status of the equipment,
A camera that captures the area including the worker,
A worker position / orientation determination unit that determines the position and orientation of the worker from the video data captured by the camera.
The operating status of the equipment and
A work classification unit that classifies setup change work based on the combination of the position and orientation of the worker,
Has a work classification system.
The setup change work is a work when one product is produced or processed.
A predetermined range within the shooting range of the camera and in the vicinity of the equipment is set as the first area.
The work classification unit classifies the setup change work as an internal setup work when the worker is present in the first area and faces the equipment while the equipment is stopped, and the equipment is in operation. When the worker is not facing the equipment in the first area, it is classified as an external setup work, and the worker is not facing the equipment in the first area while the equipment is stopped. The work classification system according to claim 1, wherein the work classification system is classified as waste when it is outside the first area, and is classified as absent when the worker is outside the first area while the equipment is in operation.
The setup change work is a work when one product is produced or processed.
A predetermined range within the shooting range of the camera and in the vicinity of the equipment is defined as a first region, and a range outside the first region is defined as a second region.
The work classification unit classifies the setup change work as an internal setup work when the worker is present in the first area and faces the equipment while the equipment is stopped, and the equipment is in operation. When the worker is not facing the equipment in the first area and when the worker is present in the second area, it is classified as an external setup work, and the worker is in the first area while the equipment is stopped. If the equipment is not facing the facility or is outside the first area, it is classified as waste, and if the worker is outside the first area and the second area while the facility is in operation, it is regarded as absent. The work classification system according to claim 1, wherein the work classification system is used for classification.
Further having a worker identification unit for identifying the worker,
The work classification system according to any one of claims 1 to 3, wherein the work classification unit associates the classified classification results with each of the workers identified by the worker identification unit.
Claims 1 to 4, wherein the worker position / orientation determination unit extracts coordinate values of a predetermined joint or skeleton of the worker from the video data, and determines the position and orientation of the worker from the coordinate values. The work classification system described in any one of.
A video recording unit that records the video data and
The work classification system according to any one of claims 1 to 5, further comprising a classification result classified by the work classification unit and a linking unit that links the video data.
The linking unit further links the operating state acquired by the equipment operating state acquisition unit and the position and orientation of the worker determined by the worker position / orientation determination unit to the classification result and the video data. , The work classification system according to claim 6.
The stage (a) of acquiring the operating status of the equipment and
The step (b) of determining the position and orientation of the worker from the video data in which the range including the worker is captured, and
The stage (c) of classifying the setup change work based on the combination of the operating state of the equipment and the position and orientation of the worker, and
A work classification program that allows a computer to execute.
The setup change work is a work when one product is produced or processed.
In the step (c), a predetermined range in the vicinity of the equipment is set as the first region within the range in which the video data is captured, and the setup change work is performed by the worker in the first region while the equipment is stopped. It is classified as an inner setup work when it exists inside and faces the equipment, and when the worker is not facing the equipment in the first area while the equipment is in operation, the outer setup work. When the equipment is stopped and the worker is not facing the equipment in the first area or outside the first area, it is classified as waste, and the worker is operating the equipment. The work classification program according to claim 8, wherein the work classification program is classified as absent when it is outside the first area.
The setup change work is a work when one product is produced or processed.
In the step (c), the setup change operation is performed by setting a predetermined range in the vicinity of the equipment as a first region and a range outside the first region as a second region within the range in which the video data is captured. Is classified as an internal setup work when the worker is present in the first area while the equipment is stopped and is facing the equipment, and the worker is in the first area while the equipment is in operation. When the work is not facing the equipment and when the work is present in the second area, it is classified as an external setup work, and the worker is not facing the equipment in the first area while the equipment is stopped. The work according to claim 8, wherein the work is classified as waste when the case is outside the first area, and is classified as absent when the worker is outside the first area and the second area while the equipment is in operation. Classification program.
The step (d) of identifying the worker and
The work classification program according to any one of claims 8 to 10, further comprising a step (e) of associating the classification results classified according to the step (c) with each identified worker.
The step (b) is any one of claims 8 to 11, wherein the coordinate value of a predetermined joint or skeleton of the worker is extracted from the video data, and the position and orientation of the worker are determined from the coordinate value. The work classification program described in one.
The step (f) of recording the video data and
The work classification program according to any one of claims 8 to 12, further comprising a classification result classified by the step (c) and a step (g) for associating the video data.
The work classification program according to claim 13, wherein the step (g) further links the operating state, the position and orientation of the worker, to the classification result and the video data.