JP2024047930A - Program, method, work management device, and work management system - Google Patents

Abstract

[Problem] To easily determine whether a worker has performed a cleaning or other task appropriately. [Solution] A program that causes a computer to execute the following: acquiring a plurality of images taken at each of a plurality of timings, including a user who is instructed to perform a task on a target work area of at least one work area, and a work tool held by the user; detecting the position of the work tool in the image based on each of the plurality of images; detecting an execution area, which is a work area currently being performed, of the at least one work area based on each of the plurality of images; calculating a task execution time, which is the time the user performed the task on the target work area, based on the target work area, the position, and the execution area; and referring to a standard task time associated with each of the work areas, determining that the task on the target work area has been completed if the task execution time satisfies a condition based on the standard task time. [Selected Figure] Figure 2

Description

Application for application of Article 30, Paragraph 2 of the Patent Act filed. Disclosure by posting on a website, posting date: March 7, 2022, website address: https://corp.avilen.co.jp/news/641818/ Disclosure at an exhibition, exhibition name: FOODEX JAPAN 2022, exhibition date: March 8, 2022 (exhibition period: March 8, 2022 - March 11, 2022)

The present invention relates to a program, a method, a work management device, and a work management system.

In factories that manufacture food, medicine, or semiconductors, clean rooms are often used to prevent foreign objects such as dust and dirt from contaminating the products. Workers who work in clean rooms often wear clean room clothing such as dustproof suits when entering the clean room. Workers clean their dustproof suits before entering the clean room.

Cleaning work is performed using, for example, an adhesive cleaner or an air shower. Workers are required to thoroughly clean in accordance with the cleaning procedures established by the manager before entering the clean room. As a method for managing users who are subject to conditions for entry, Patent Document 1 shows an entry permission management device that prohibits workers who manufacture electronic components from entering the electronic component manufacturing line while carrying static electricity.

JP 2017-162741 A

In the invention described in Patent Document 1, the static elimination state of a worker is judged based on the contact resistance value measured by a contact resistance meter. Unlike the static elimination state, which can be measured electrically, it is difficult to directly measure the degree of adhesion of foreign matter such as dust and dirt that adheres to the special clothing worn when entering a clean room. Even in such cases, it is required to be able to judge whether or not a worker entering a clean room has performed cleaning work appropriately and to be able to supervise the cleaning work.

The present invention aims to provide a program, method, work management device, and work management system that can easily determine whether a worker has performed a cleaning or other task appropriately.

A program according to one aspect of the present invention causes a computer to execute an image acquisition process that acquires a plurality of images, each captured at a plurality of timings, that include a user who is instructed to perform a task at a target work area of at least one work area and a work tool held by the user, a position detection process that detects the position of the work tool in the image based on each of the plurality of images, and an execution part detection process that detects an execution part, which is a work area at which the user is performing a task using the work tool, of the at least one work area based on each of the plurality of images.

The program also causes the computer to execute an execution time calculation process that calculates the work execution time, which is the time it takes for the user to perform work on the target work area, based on the target work area, position, and execution area, and a completion determination process that refers to the standard work time associated with each work area and determines that the work on the target work area has been completed if the work execution time satisfies a condition based on the standard work time.

According to this aspect, the position of the work tool can be used to calculate the work execution time during which the user performed work on the target work area, and based on the calculated work execution time, it can be detected whether the work on the target work area has been completed. This makes it possible to determine, for example, whether the worker has performed work such as cleaning appropriately in a simple manner without directly detecting minute foreign objects adhering to special clothing.

In the program of the above aspect, the multiple images may include a first image taken at a first timing and a second image taken at a second timing after the first timing. Furthermore, the position detection process may detect a first position of the work tool in the first image based on the first image, and detect a second position of the work tool in the second image based on the second image, and the performance time calculation process may calculate the work performance time by adding the time between the second timing and the first timing to the work performance time when the second position is different from the first position.

According to this aspect, the work execution time can be calculated based on the change in the position of the work tool at different times. The first position and the second position being the same means that the work tool is not being moved and a work operation is not being performed. The first position and the second position being different means that the work tool is being moved and a work operation is being performed. In this way, in work such as cleaning, where the execution of a work operation can be determined based on the position of the work tool, a simple method can be used to determine whether the worker has performed the work appropriately.

The program of the above aspect may further cause the computer to execute an identification information acquisition process for acquiring identification information for identifying a user, and a result recording process for, when it is determined that the work on the target work area has been completed, recording the identification information in association with information indicating that the work on the target work area has been completed.

According to this aspect, the completion of work for each target work area is recorded. By having a manager at the factory or the like check the recorded information, it becomes possible to grasp the work status of the workers and supervise the work.

In the program of the above aspect, the result recording process may record, when work on all target work areas is completed, the identification information and information indicating that the work by the user has been completed in association with each other.

According to this aspect, the completion of a series of tasks such as cleaning performed by a certain worker (user) is recorded. By having a manager of a factory or the like check the recorded information, it becomes possible, for example, to determine whether each worker is performing the tasks appropriately, and to identify the worker in the event of contamination during production, enabling the manager to properly supervise the tasks.

The program of the above aspect may further cause the computer to execute a screen information generation process that generates instruction screen information for displaying an instruction screen indicating the target work area to the user.

According to this aspect, the target work area on which the user is to work is displayed to the user. The user can perform work such as cleaning while checking the instruction screen, which allows the user to proceed with the work appropriately.

In the program of the above aspect, the screen information generation process may generate instruction screen information that includes a captured image of the user.

According to this aspect, the user's own condition while working is displayed to the user. The user can perform tasks such as cleaning while checking their own condition on the instruction screen, so the user can proceed with the work appropriately.

In the program of the above aspect, the screen information generation process may generate instruction screen information including progress information indicating the progress of work on the target work area based on the reference work time and the work execution time.

According to this aspect, the progress of the work is displayed to the user through the instruction screen. The user can perform work such as cleaning while checking the progress. For example, if the progress is not satisfactory, the user can modify the work method and proceed with the work appropriately.

In the program of the above aspect, at least one work site includes a first work site and a second work site where work is performed next to the first work site, and the screen information generation process generates information for displaying to the user an instruction screen showing the first work site as the target work site, and when it is determined in the completion determination process that the work of the first work site as the execution site has been completed, instruction screen information may be generated that shows the second work site as the target work site.

According to this aspect, when a user has finished working on a first work area, the user is continuously instructed to work on a second work area. This allows the user to be instructed to work on a series of work areas in succession, reducing the number of times the user misses a work area and allowing the user to proceed with the work appropriately.

A method according to another aspect of the present invention includes a computer acquiring a plurality of images taken at each of a plurality of timings, the images including a user who is instructed to perform a task at a target work area of at least one work area and a work tool held by the user, detecting the position of the work tool in the image based on each of the plurality of images, and detecting an execution area, which is a work area where the user is performing a task using the work tool, of the at least one work area based on each of the plurality of images.

The method further includes calculating a task execution time, which is the time it takes for the user to perform the task on the task part, based on the target task part, the position, and the task part, and referring to a standard task time associated with each task part, determining that the task on the task part has been completed if the task execution time satisfies a condition based on the standard task time.

A work management device according to another aspect of the present invention includes an image acquisition unit that acquires a plurality of images taken at each of a plurality of timings, the images including a user who is instructed to perform a task at a target work area of at least one work area and a work tool held by the user, a position detection unit that detects the position of the work tool in the image based on each of the plurality of images, and an execution area detection unit that detects an execution area, which is a work area where the user is performing a task using the work tool, of the at least one work area based on each of the plurality of images.

The work management device further includes an execution time calculation unit that calculates the work execution time, which is the time it takes for the user to perform work on the execution part, based on the target work part, the position, and the execution part, and a completion determination unit that refers to the standard work time associated with each of the work parts and determines that the work on the execution part is completed if the work execution time satisfies a condition based on the standard work time.

A work management system according to another aspect of the present invention includes a display unit that displays an instruction screen that indicates to a user which of at least one work area the user is to work on, an imaging unit that captures a plurality of images taken at a plurality of timings including the user and a work tool held by the user, and a work management device.

In the above-described aspect of the work management system, the work management device includes an image acquisition unit that acquires a plurality of images from an imaging unit, a position detection unit that detects the position of the work tool in the image based on each of the plurality of images, and an execution part detection unit that detects, among at least one work part, an execution part that is a work part where a user is working using the work tool, based on each of the plurality of images.

The work management device further includes an execution time calculation unit that calculates a work execution time, which is the time it takes for the user to perform work on the work part, based on the work part and the position; a completion determination unit that refers to a standard work time associated with each work part and determines that the work on the work part is completed if the work execution time satisfies a condition based on the standard work time; and a screen information generation unit that generates information for displaying to the user an instruction screen indicating the target work part on which the user should perform work.

The present invention provides a program, method, work management device, and work management system that can easily determine whether a worker has performed a cleaning or other task appropriately.

1 is a schematic diagram of a work management system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an overview of a process performed by the work management system according to the present embodiment. FIG. 2 is a diagram illustrating an example of a hardware configuration of a work management apparatus 101. 1 is a block diagram of a work management device according to an embodiment of the present invention. 4 is an example of user information according to the embodiment. 4 is an example of work area information according to the present embodiment. 4 is an example of work management information according to the embodiment. 11 is another example of work management information according to the embodiment. 5 is an example of work result information according to the embodiment. 4 is a flowchart illustrating a process performed by the work management device according to the present embodiment. 11 is a flowchart illustrating a screen information generating process according to the embodiment. FIG. 11 is a diagram illustrating an example of an instruction screen according to the embodiment. 10A to 10C are diagrams illustrating an overview of a position detection process and an execution part detection process according to the present embodiment. 11 is a flowchart illustrating an implementation time calculation process according to the present embodiment. 11A and 11B are diagrams illustrating an example of updating an instruction screen according to the embodiment.

A preferred embodiment of the present invention will be described with reference to the attached drawings. In each drawing, the same reference numerals are used to denote the same or similar configurations.

FIG. 1 is a block diagram showing an outline of a work management system 10 according to this embodiment. The work management system 10 includes a work management device 101 and an imaging device 201. The work management device 101 and the imaging device 201 are connected via a wired or wireless connection so as to be able to send and receive information.

In this embodiment, cleaning work performed in a food factory or the like will be described as an example of work. The work management system 10 is a system in which the work management device 101 determines whether or not the cleaning work of the cleaning area (work area) to be cleaned by the user has been completed based on a video (multiple images) of the worker (user) captured by the imaging device 201. In this cleaning work, a user wearing clean room clothing cleans the clothing he or she is wearing using an adhesive cleaner (work tool).

The work management device 101 is an information processing device having a computer that performs a predetermined process by executing a predetermined program. The work management device 101 is, for example, a tablet terminal, a smartphone, a personal computer, etc. A program that performs the functions of the work management device 101 is installed in an information processing device such as a tablet terminal, and is executed by the CPU of the work management device 101, causing the information processing device to function as the work management device 101.

The imaging device 201 is a device such as a camera capable of capturing video of a user. In this embodiment, the images captured by the imaging device 201 are described as still images constituting a video of a user's actions. In other words, the multiple images are images in each frame of the video captured by the imaging device 201. The imaging device 201 may be a camera provided inside the work management device 101, and the work management device 101 and the imaging device 201 do not necessarily have to be separate devices.

With reference to FIG. 2, an overview of information processing by the work management system 10 will be described. In the example of FIG. 2, a user U holds a cleaning tool C in his or her hand and performs cleaning work. For example, the user checks a target work area indicating an area to be cleaned by the user on the screen of the work management device 101, and cleans the target work area. The imaging device 201 captures multiple images including the user U and the cleaning tool C as a video. The video captured by the imaging device 201 is transmitted to the work management device 101.

Specifically, as shown in step (A1) of FIG. 2, an image IG1 including a user U in a certain posture and a cleaning tool C is acquired by the work management device 101 via the imaging device 201.

In step (B1), the work management device 101 performs a position detection process and a working part detection process on the image IG1. The position detection process detects the position of the user U and the position of the cleaning tool C in the image IG1. The working part detection process detects the cleaning part where the user U is performing the cleaning work as the working part. Based on the image IG1, the position of the user U and the position of the cleaning tool C in the image IG1 are detected to be positions P11 and P12, respectively, which are defined by bounding boxes. Also, based on the image IG1, the working part of the user U shown in the image IG1 is detected to be the "shoulder". The position detection process and the working part detection process are performed using, for example, a learning model learned by machine learning.

Similarly, as shown in step (A2), image IG2 is acquired at a timing later than the timing at which image IG1 is acquired. Also, in step (B2) at a timing later than step (B1), the positions of user U and cleaning tool C in image IG1 are detected as positions P21 and P22, respectively, and the performing area is detected as the "shoulder."

In step (C), the work management device 101 calculates the work execution time for the "shoulder" using positions P12 and P22 of the cleaning tool C detected in images IG1 and IG2. At this time, the work management device 101 calculates the work execution time by adding up the work execution time when positions P12 and P22 are different.

In step (D), the work management device 101 compares the calculated work execution time with a predetermined standard work time. For example, if the work execution time is equal to or longer than the standard work time, in step (E), the work management device 101 determines that the cleaning work for the execution area has been completed.

An example of the hardware configuration of the work management device 101 will be described with reference to FIG. 3. The work management device 101 has a processor 301 such as a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit), a storage device 302 such as a memory, a HDD (Hard Disk Drive) and/or an SSD (Solid State Drive), a communication IF (Interface) 303 for wired or wireless communication, an input device 304 for accepting input operations, and an output device 305 for outputting information. The input device 304 is, for example, a keyboard, a touch panel, a mouse, and/or a microphone. The output device 305 is, for example, a display, a touch panel, and/or a speaker.

The functional block configuration of the work management device 101 will be described with reference to FIG. 4. The work management device 101 has a communication unit 1011, a display unit 1012, a memory unit 1013, an image acquisition unit 1014, an operation part detection unit 1015, a position detection unit 1016, an implementation time calculation unit 1017, a completion determination unit 1018, a screen information generation unit 1019, an identification information acquisition unit 10110, a result recording unit 10111, and a work part determination unit 10112.

The communication unit 1011 can be realized using the communication IF 303 provided in the work management device 101. The display unit 1012 can be realized using the output device 305 provided in the work management device 101. The memory unit 1013 can be realized using the storage device 302 provided in the work management device 101. The image acquisition unit 1014, the execution part detection unit 1015, the position detection unit 1016, the implementation time calculation unit 1017, the completion determination unit 1018, the screen information generation unit 1019, the identification information acquisition unit 10110, the result recording unit 10111, and the work part determination unit 10112 can be realized by the processor 301 of the work management device 101 executing a program stored in the storage device 302. In addition, the program can be stored in a storage medium. The storage medium storing the program may be a non-transitory storage medium (Non-transitory computer readable medium). The non-transitory storage medium is not particularly limited, but may be, for example, a storage medium such as a USB memory or a CD-ROM.

The communication unit 1011 controls communication between the work management device 101, which includes the imaging device 201, and an external information processing device.

The display unit 1012 is, for example, a display of a tablet terminal, a smartphone, etc., and displays a screen to the user.

The memory unit 1013 stores various information used for processing in the work management device 101. The memory unit 1013 stores a user DB 10131, a work part DB 10132, a work management DB 10133, a work result DB 10134, an execution part detection model 10135, and a position detection model 10136.

The user information stored in user DB 10131 will be described with reference to Figure 5. User information has the fields "user name" and "user ID". In the "user name" field, the name of the user who uses the work management system 10 is stored, for example, "user A". In the "user ID" field, information identifying an individual user is stored, for example, "USER01".

The work part information stored in the work part DB10132 will be described with reference to Figure 6. The work part information has the fields "cleaning part name", "cleaning order", "reference work time", and "reference operation video".

The information stored in each of the fields "Cleaning area name," "Cleaning order," "Reference work time," and "Reference operation video" is set by the manager of the work management system 10 (e.g., the operations manager of a food factory). The manager can set the work area information as desired depending on the characteristics of the factory under his/her management, and an example is shown in FIG. 6.

The "Name of Cleaning Part" field stores the cleaning parts that the user should clean. The cleaning parts may be distinguished between front and back, for example, head (front), head (rear), neck (front), and neck (rear). Also, the cleaning parts may be distinguished between left and right, for example, right arm, left arm, right leg, and left leg.

The "cleaning order" field stores the order in which the user cleans each cleaning area. The order is set, for example, so that the cleaning areas move from top to bottom, from the user's head to the user's toes. The "reference work time" field stores the time that the user should perform the cleaning action to clean each cleaning area. The "reference operation video" field stores information indicating a video for explaining the cleaning method of each cleaning area. The "reference operation video" is, for example, information indicating the address in the storage unit 1013 of a video file stored in the storage unit 1013. Note that, when the work management device 101 connects to an external information processing device via the communication unit 1011 to acquire a video file, the information stored in the "reference operation video" field may be information indicating the access destination.

Specifically, for example, if the "name of cleaning area" is "shoulders," its "cleaning order" is "5th" and its "standard work time" is "8 seconds." In addition, the "standard operation video" for cleaning the "shoulders" is stored as "file T5."

The work management information stored in the work management DB 10133 will be described with reference to Figure 7. The work management information is information that is updated according to the progress of each user when the user performs cleaning work using the work management system 10. The work area information has the fields "performing user", "cleaning area name", "work performance time", "reference work time", and "cleaning status".

The "Performing User" field stores the name of the user who used the work management system 10. The "Cleaning Area Name" field stores the name of the cleaning area that the user has cleaned or will clean. The information stored in the "Cleaning Area Name" field is arranged and stored according to the "Cleaning Order" of the work area DB 10132. The "Work Performance Time" field stores the time that the user performed the cleaning operation for each area, calculated by the work management device 101. The "Reference Work Time" field stores the time that the user should perform the cleaning operation. The "Cleaning Status" field stores the status of the user's cleaning of each cleaning area, as "Completed," "Cleaning in Progress," or "Incomplete."

For example, in FIG. 7, the task execution time for user A in regards to the cleaning area "head (front)" is stored as "7 seconds" and the cleaning status is stored as "completed." This indicates that user A has completed cleaning the cleaning area of the head (front). In FIG. 7, user A has completed cleaning the cleaning areas from "head (front)" to "neck (rear)."

In FIG. 7, it is stored that user A is "cleaning" the "shoulder" cleaning area. The area with a cleaning status of "cleaning" is the target work area. The work execution time is updated by the work management device 101 according to user A's cleaning action. When the work execution time reaches or exceeds the standard work time corresponding to each area, the cleaning status is updated from "cleaning" to "completed."

Figure 8 is an example of work management information after user A has completed cleaning the "shoulders." In Figure 8, user A has cleaned the "shoulders" area for nine seconds, and the cleaning status has been updated to "completed." At this time, user A is currently cleaning the "chest" area.

9, the work result information stored in the work result DB 10134 will be described. The work result information has the following items: "user name", "user ID", "completion determination result", "implementation date and time", and "recorded video". The "completion determination result" item stores information indicating whether each user has completed cleaning all cleaning areas based on the work management information stored in the cleaning management DB 10133. The "implementation date and time" item stores the date and time when user A has completed cleaning all cleaning areas. The information stored in the "implementation date and time" item may be the time when cleaning of all cleaning areas is completed, or the time when cleaning is started. The "recorded video" item stores information indicating a video in which a series of cleaning videos by a user is recorded. The "recorded video" is, for example, information indicating the address in the storage unit 1013 of a recording file stored in the storage unit 1013. Note that the information stored in the "recorded video" item may be information indicating the access destination when the work management device 101 connects to an external information processing device through the communication unit 1011 and stores a video file.

In the example of FIG. 9, user A (user ID: USER01) has "completed" cleaning of all cleaning areas, the "implementation date and time" is stored as "2022/7/5 11:55:30", and the "recorded video" is stored as "File A". Also, as shown for user C, the "completion determination result" field may store information of "incomplete" in advance, and the information stored in the "completion determination result" field may be updated according to processing by the completion determination unit 1018 and result recording unit 10111 described below. Also, when the "completion determination result" is "incomplete", no information is stored in the "implementation date and time" and "recorded video" fields.

The work management information and work result information do not need to be stored in the work management device 101 itself, but may be stored, for example, in an external computer that can communicate with the work management device 101. In this case, the work management device 101 transmits information for updating the work management information and work result information to the external computer via the communication unit 1011.

The execution part detection model 10135 is a mathematical model that detects the execution part, which is the part where the user is currently cleaning, based on the image of the user acquired by the image acquisition unit 1014. As the execution part detection model 10135, for example, a learning model using a neural network such as Efficient Net can be used.

The position detection model 10136 is a mathematical model that detects an object in an image and detects its position based on an image of a user acquired by the image acquisition unit 1014 described below. As the position detection model 10136, for example, a learning model capable of real-time object detection and position detection such as YOLO can be used.

The image acquisition unit 1014 acquires a video captured by the imaging device 201 from the imaging device 201. The captured video shows the user and the cleaning tool while cleaning. In other words, the image acquisition unit 1014 acquires multiple images captured at multiple times as a video.

The performing part detection unit 1015 detects the performing part, which is the part being cleaned by the user using the cleaning tool, by inputting the image acquired by the image acquisition unit 1014 into the performing part detection model 10135 and acquiring the user's cleaning action. Here, when detecting the performing part, the performing part detection unit 1015 may use the position of the cleaning tool detected by the position detection unit 1016. For example, the position detection unit 1016 can detect that the right arm or left arm is the performing part based on the position of the cleaning tool in the image. This makes it possible to further distinguish between left and right, for example, when the performing part detection model 10135 can only detect the cleaning action of the "arms."

The position detection unit 1016 inputs the image acquired by the image acquisition unit 1014 into the position detection model 10136 and acquires the position of the object in the image, thereby detecting the position of the cleaning tool in the image. The position detection unit 1016 may also acquire the position of the user.

The execution time calculation unit 1017 calculates the task execution time, which is the time it takes for the user to clean the target task area, based on the target task area to be cleaned by the user, the position of the cleaning tool, and the execution area. While the user is cleaning, the execution time calculation unit 1017 calculates the task execution time according to the user's actions, and updates the information stored in the "task execution time" field in the task management information of the cleaning management DB 10133.

The completion determination unit 1018 references the "reference work time" of the target work area in the work area information of the work area DB 10132. The completion determination unit 1018 determines that cleaning of the target work area is completed when the work implementation time satisfies a condition based on the referenced reference work time. Here, the condition based on the reference work time is, for example, "the work implementation time is equal to or longer than the reference work time" or "the difference between the work implementation time and the reference work time is equal to or less than a specified value."

The screen information generating unit 1019 generates information for displaying on the display unit 1012 an instruction screen that shows the user the target work area, an image (video) of the user cleaning, progress information showing the progress of cleaning, and the like. The screen information generating unit 1019 references the work management information in the cleaning management DB 10133 to acquire the target work area to be cleaned. The screen information generating unit 1019 also references the image including the user and cleaning tools acquired by the image acquiring unit 1014. The screen information generating unit 1019 references the work management information in the cleaning management DB 10133 to acquire the work execution time and standard reference time for the cleaning area being cleaned. The screen information generating unit 1019 generates progress information based on the acquired work execution time and standard reference time. Specific examples of screens will be described later.

The identification information acquisition unit 10110 acquires identification information for identifying a user from a user through an input device for inputting information to the work management device 101. The identification information is, for example, a user ID stored in the user DB 10131. In addition, the input device may be, for example, the display unit 1012 that functions as a touch panel when the work management device 101 is a tablet terminal, or may be a terminal such as a keyboard connected to the work management device 101.

The result recording unit 10111 updates the information stored in the "cleaning status" field of the work management information in the cleaning management DB 10133 based on the determination result of the completion determination unit 1018. The result recording unit 10111 also references the work management information and updates the information stored in the "completion determination result" field of the work result information in the work result DB 10134. For example, when the cleaning status of all cleaning areas that a user is to clean is "completed" in the work management information, the result recording unit 10111 updates the information stored in the "completion determination result" field associated with the user to "completed". The result recording unit 10111 also updates the information stored in the "implementation date and time" field based on the date and time when the information stored in the "completion determination result" field was updated to "completed".

The processing in the work management device 101 will be described with reference to Figure 10.

In step S1001, the identification information acquisition unit 10110 acquires the user ID input by the user performing the cleaning work.

In step S1002, the work area determination unit 10112 refers to the work area DB 10132 and obtains work area information.

In step S1003, the work area determination unit 10112 determines the target work area that the user should clean according to the cleaning order of the work area information. For example, the first target work area is the "head (front)" according to the work area information in FIG. 6.

In step S1004, the screen information generation unit 1019 performs a screen information generation process. The screen information generation process is described with reference to FIG. 11.

In step S1101, the screen information generating unit 1019 acquires the image of the user acquired by the image acquiring unit 1014.

In step S1102, the screen information generating unit 1019 acquires the target work area determined by the work area determining unit 10112.

In step S1103, the screen information generating unit 1019 refers to the work part DB 10132 and obtains the standard work time associated with the target work part. For example, if the target work part is the "head (front)", the standard work time is "5 seconds", and if the target work part is the "shoulder", the standard work time is "8 seconds".

In step S1104, the screen information generating unit 1019 refers to the cleaning management DB 10133 and obtains the work execution time associated with the target work area.

In step S1105, the screen information generating unit 1019 generates progress information indicating the progress of cleaning the target work area based on the reference work time and the work execution time. The progress information is, for example, information for displaying the ratio of the work execution time to the reference work time, or information for displaying the difference in seconds between the remaining work execution time and the reference work time.

In step S1106, the screen information generating unit 1019 refers to the work area DB 10132 and obtains the reference movement video associated with the target work area.

In step S1107, the screen information generating unit 1019 generates instruction screen information based on the user's image, the target work area, the progress information, and the reference operation video.

An example of the instruction screen will be described with reference to FIG. 12. The instruction screen 1200 shown in FIG. 12 has a target work area display field 1201, a progress information display area 1202, a work execution time display field 1203, an icon 1204, a reference operation display area 1205, and a user display area 1206.

In the target work area display field 1201, information indicating the target work area is displayed. Here, "shoulders" are shown as the target work area.

Progress information is displayed in the progress information display area 1202. Here, progress information generated based on the work management information in FIG. 7 is shown. In FIG. 7, the shoulder work time is "5 seconds" and the reference work time is "8 seconds", so in FIG. 12, a progress display section 12021 showing the ratio of the work time to the reference work time is displayed in the progress information display area 1202. In addition, the work time itself may be displayed in the work time display field 1203 as information that visually displays the work time.

Icon 1204 is an icon for indicating the target work area, and is generated by screen information generating unit 1019 with reference to memory unit 1013. Icon 1204 makes it possible to visually indicate the target work area to the user. In icon 1204, the area display area 12041 corresponding to the target work area is highlighted. This allows the user to visually grasp the target work area.

The reference action video is displayed in the reference action display area 1205. This allows the user to perform cleaning actions while checking the model cleaning actions, resulting in more appropriate cleaning.

The user display area 1206 displays an image (video) of the user. This allows the user to check their own cleaning actions while performing the cleaning operation, resulting in more appropriate cleaning.

Returning to FIG. 10, the process following the screen information generation process described above will be described. In step S1005, the display unit 1012 displays an instruction screen based on the instruction screen information. The user who has been shown the instruction screen performs a cleaning operation.

In step S1006, the image acquisition unit 1014 acquires a first image at a first timing when the user is cleaning. The first image is, for example, image IG1 in FIG. 13.

In step S1007, the cleaning part detection unit 1015 detects the cleaning part, which is the part being cleaned by the user, based on the first image. For example, the cleaning part is the "shoulder" based on the image IG1 in FIG. 13.

In step S1008, the position detection unit 1016 detects a first position of the cleaning tool in the first image based on the first image. For example, based on image IG1 in FIG. 13, the first position is position P1. More specifically, position P1 is two coordinates that define a bounding box that surrounds the cleaning tool, for example, coordinates indicating the lower left and upper right vertices of a rectangle. Based on image IG1, position P1 is detected as {(X1a, Y1a), (X1b, Y1b)}.

In step S1009, the image acquisition unit 1014 acquires a second image at a second timing that is after the first timing. The second image is, for example, image IG2 in FIG. 13.

In step S1010, the execution part detection unit 1015 detects the execution part based on the second image. For example, the execution part is the "shoulder" based on image IG2 in FIG. 13.

In step S1011, the position detection unit 1016 detects a second position of the cleaning tool in the second image based on the second image. For example, the second position is position P2 based on image IG2 in FIG. 13. Based on image IG2, position P2 is detected as {(X2a, Y2a), (X2b, Y2b)}.

In step S1012, the implementation time calculation unit 1017 performs implementation time calculation processing. The implementation time calculation processing is performed by comparing positions P1 and P2 described with reference to FIG. 13. The implementation time calculation processing will be described in detail with reference to FIG. 14. Here, a case will be described in which the task implementation time of a user who is cleaning the shoulder as the target task part is calculated, similarly to FIG. 13.

In step S1401, the implementation time calculation unit 1017 acquires the first implementation part based on the first image. Based on image IG1, the first implementation part is the shoulder.

In step S1402, the implementation time calculation unit 1017 acquires the second implementation part based on the second image. Based on image IG2, the second implementation part is also the shoulder.

In step S1403, the execution time calculation unit 1017 refers to the target work area determined by the work area determination unit 10112, and detects whether the first execution area and the second execution area are target work areas. If a negative judgment is made in step S1403, that is, if either the first execution area or the second execution area is not a target work area, it is determined that cleaning has not been performed, and the process of adding up the work execution time is terminated.

In the example of Figure 13, the first and second execution parts are both the shoulder, which is the target work part, so a positive determination is made in step S1403.

In step S1404, the implementation time calculation unit 1017 acquires the first position detected by the position detection unit 1016.

In step S1405, the implementation time calculation unit 1017 acquires the second position detected by the position detection unit 1016.

In step S1406, the implementation time calculation unit 1017 detects whether the first position and the second position are different. If the first position and the second position are the same, it means that the cleaning tool is not being moved, i.e., the sticky cleaner is not being moved along the user's clothes, and the cleaning operation is not being performed. If the first position and the second position are different, it means that the cleaning tool is being moved, i.e., the sticky cleaner is being moved along the user's clothes, and the cleaning operation is being performed.

If the first position and the second position are the same, a negative determination is made in step S1406, meaning that it is determined that a cleaning operation is not being performed, and the process of adding up the work execution time ends.

In the example of FIG. 13, position P1 and position P2 are in different positions, so a positive determination is made in step S1406.

In step S1407, the implementation time calculation unit 1017 adds the work implementation time. More specifically, the implementation time calculation unit 1017 adds the time between the first timing and the second timing to the work implementation time of the target work area. The implementation time calculation unit 1017 updates the work implementation time of the cleaning area being cleaned in the work management information based on the calculated work implementation time.

Returning to FIG. 10, the process following the above-mentioned execution time calculation process will be described. In step S1013, the completion determination unit 1018 refers to the reference work time for the target work area.

In step S1014, the completion determination unit 1018 determines whether the task execution time is equal to or longer than the reference task time. If a negative determination is made in step S1014, the process returns to step S1006 and the process is repeated.

If a positive judgment is made in step S1014, in step S1015, the completion judgment unit 1018 judges that cleaning of the target work area has been completed.

In step S1016, the completion determination unit 1018 updates the "cleaning status" information in the work management information for the target work area from "cleaning in progress" to "completed." An example of this update is shown in Figures 7 and 8.

In step S1017, the completion determination unit 1018 detects whether cleaning of all areas has been completed based on the work management information.

If a negative judgment is made in step S1017, in step S1018, the work area determination unit 10112 updates the target work area based on the work area information and the work management information so that the cleaning area designated as the next cleaning order becomes the target work area. The process of step S1004 is then repeated to detect the cleaning operation of the next cleaning area.

If the answer is YES in step S1017, that is, if the user has completed cleaning all areas that need to be cleaned, the completion determination unit 1018 records in step S1019 that the cleaning has been completed. Specifically, the completion determination unit 1018 updates the "Completion Determination Result" field of the user who has completed cleaning in the work result information to "Completed" and stores information in the "Date and Time of Implementation" and "Recorded Video" fields.

An example of a screen when the target work area is updated in step S1018 will be described with reference to FIG. 15. FIG. 15(a) shows an instruction screen 1200 in which the user has been instructed to clean the shoulders and the work execution time has reached 9 seconds. At this time, since the work execution time is equal to or longer than 8 seconds, which is the standard work time for the shoulders, it is determined that shoulder cleaning has been completed.

When it is determined that cleaning is complete, as shown in FIG. 15(b), the screen information generating unit 1019 causes the display unit 1012 to display a display 1501 indicating that cleaning is complete on the instruction screen 1200. The work management information at this time is, for example, the information shown in FIG. 7.

Then, the work part determination unit 10112 updates the target work part to "chest", which is next to the shoulders in the work management information. The work management information at this time is, for example, the information shown in FIG. 8. Based on the updated target work part, the screen information generation unit 1019 causes the display unit 1012 to display the instruction screen 1200 shown in FIG. 15(c). In FIG. 15(c), the target work part displayed in the target work part display field 1201, the progress information in the progress information display area 1202, and the work execution time information in the work execution time display field 1203 are updated. In addition, the icon 1204 is updated to include a part display area 12042 indicating the chest. In addition, the video displayed in the reference action display area 1205 becomes the reference action video corresponding to the chest. The user then performs the chest cleaning operation while referring to the instruction screen 1200.

The above-described embodiments are intended to facilitate understanding of the present invention, and are not intended to limit the present invention. The elements of the embodiments, as well as their arrangement, conditions, shape, size, etc., are not limited to those exemplified, and may be modified as appropriate. Furthermore, configurations shown in different embodiments may be partially substituted or combined.

10... Work management system, 101... Work management device, 201... Imaging device, 1011... Communication unit, 1012... Display unit, 1013... Storage unit, 1014... Image acquisition unit, 1015... Work area detection unit, 1016... Position detection unit, 1017... Work time calculation unit, 1018... Completion determination unit, 1019... Screen information generation unit, 10110... Identification information acquisition unit, 10111... Result recording unit, 10112... Work area determination unit

Claims (11)

On the computer,
an image acquisition process for acquiring a plurality of images taken at each of a plurality of timings, the images including a user who is instructed to perform a task at a target work site among at least one work site and a work tool held by the user;
a position detection process for detecting a position of the work tool in each of the plurality of images;
a work part detection process for detecting a work part, which is a work part that the user is working on using the work tool, among the at least one work part based on each of the plurality of images;
an execution time calculation process for calculating an execution time of the work performed by the user on the target work area based on the target work area, the position, and the execution area;
a completion determination process that refers to a standard work time associated with each of the work areas, and determines that the work of the target work area has been completed if the work execution time satisfies a condition based on the standard work time. The program according to claim 1,
the plurality of images include a first image captured at a first timing and a second image captured at a second timing that is later than the first timing,
The position detection process includes:
detecting a first position of the work tool in the first image based on the first image, and detecting a second position of the work tool in the second image based on the second image;
The implementation time calculation process includes:
a program that calculates an operation execution time by adding a time between the second timing and the first timing to an operation execution time when the second position is different from the first position; 3. The program according to claim 1 or 2,
The computer includes:
an identification information acquisition process for acquiring identification information for identifying the user;
and a result recording process for recording, when it is determined that the work on the target work area has been completed, the identification information in association with information indicating that the work on the target work area has been completed. The program according to claim 3,
The result recording process is performed when the work of all the target work parts is completed.
A program that records the identification information in association with information indicating that the work by the user has been completed. 3. The program according to claim 1 or 2,
The computer includes:
The program further executes a screen information generating process for generating instruction screen information for displaying an instruction screen indicating the target work area to the user. The program according to claim 5,
The screen information generating process generates the instruction screen information including a captured image of the user. The program according to claim 5,
The screen information generation process generates the instruction screen information including progress information indicating the progress of work at the target work portion based on the reference work time and the work execution time. The program according to claim 5,
the at least one working position includes a first working position and a second working position on which work is performed subsequent to the first working position;
The screen information generation process generates information for displaying the instruction screen to the user, the instruction screen indicating the first work area as the target work area, and when it is determined in the completion determination process that the work of the first work area as the execution area has been completed, generates the instruction screen information indicating the second work area as the target work area. The computer
acquiring a plurality of images taken at a plurality of timings, the images including a user who is instructed to perform a task at a target work site among at least one work site, and a work tool held by the user;
detecting a position of the work tool in each of the plurality of images based on the images;
Detecting an execution part, which is a work part where the user is working using the work tool, among the at least one work part based on each of the plurality of images;
Calculating a task execution time, which is a time during which the user performed a task on the task part, based on the target task part, the position, and the task part;
referencing a standard work time associated with each of the work portions, and determining that the work of the work portion has been completed if the work execution time satisfies a condition based on the standard work time. an image acquisition unit that acquires a plurality of images captured at a plurality of timings, the images including a user who is instructed to perform a task at a target work site among at least one work site and a work tool held by the user;
a position detection unit that detects a position of the work tool in each of the plurality of images based on the images;
a work part detection unit that detects, from among the at least one work part, a work part that is a work part being performed by the user using the work tool, based on each of the plurality of images;
an execution time calculation unit that calculates an execution time, which is a time during which the user performs an operation on the execution part, based on the target work part, the position, and the execution part;
A work management device comprising: a completion determination unit that refers to a standard work time associated with each of the work parts, and determines that the work of the execution part is completed if the work execution time satisfies a condition based on the standard work time. a display unit that displays an instruction screen that indicates to a user a target work area on which the user is to work among at least one work area;
An imaging unit that includes the user and a work tool held by the user, and captures a plurality of images captured at a plurality of timings;
A work management device,
an image acquisition unit that acquires the plurality of images from the imaging unit;
a position detection unit that detects a position of the work tool in each of the plurality of images based on the images;
a work part detection unit that detects, from among the at least one work part, a work part that is a work part being performed by the user using the work tool, based on each of the plurality of images;
an execution time calculation unit that calculates an execution time, which is a time during which the user performs an operation on the execution part, based on the target work part, the execution part, and the position;
a completion determination unit that refers to a standard work time associated with each of the work parts, and determines that the work of the work parts is completed when the work execution time satisfies a condition based on the standard work time;
a screen information generating unit that generates information for displaying to the user an instruction screen that indicates a target work area on which the user is to work; and
A work management system comprising:

Images