JP2022111778A - Operation support system, operation support control device, and operation support control program - Google Patents

Abstract

To check work procedure of a next item before actual work of the next item to establish work of a work site of a one-person system when a series of operation is sequentially executed.SOLUTION: Work support control is made to be conducted by performing communication between a smartphone 12 and smart glasses 10 while the commercially available smart glasses 10 is applied as a wearable communication terminal of the work support control system and a work support control program that is characteristics of the present embodiment is installed as an application program of the smartphone 12. For this reason, it is not necessary to develop or purchase (or sign a lease of) smart glasses having a special function, for example, and it is possible to easily perform work support control in accordance with various work procedures for general purposes. Further, a worker 28 can eliminate complexity such as to work with both hands while holding a work procedure manual or the like with one hand.SELECTED DRAWING: Figure 1

Description

The present invention relates to a work support system, a work support control device, and a work support control program that support the work of a worker working at a work site based on the progress of the work.

At the work site, in order to prevent work mistakes, etc., work is carried out by a two-person system: a person who executes the work and a confirmation person who checks the progress of the work while comparing the actual work with the work procedure manual. is preferred. In order to realize a two-person system, it is becoming complicated and difficult to allocate a limited number of workers, and although work safety and accuracy are ensured, work efficiency such as work delays may lead to a decrease in

Patent Literature 1 describes that a work procedure corresponding to a trouble that has occurred is presented to a worker at an installation location of a water treatment facility. More specifically, the maintenance support device for water treatment equipment includes a storage unit that stores work procedure data describing a plurality of work procedures for coping with troubles in the water treatment equipment together with the work level of each work; A determination unit that determines whether or not there is a problem in the water treatment facility based on the signal, identifies work to deal with the problem if there is a problem, and determines whether the work level of the identified work is at or below a predetermined level. and a transmission unit that, when the work level of the specified work is equal to or lower than a predetermined level, extracts the work procedure data of the work from the storage unit and transmits the data to the reproduction device.

Patent Literature 2 describes providing a worker with information for appropriately adjusting work. More specifically, input information including information indicating the work status of the first worker is obtained from the input device of the first worker, and the work list is updated in association with the work status information indicating the work status of the first worker. Then, the work status information included in the updated work list is transmitted to the second worker's augmented reality device. In addition, in this patent document 2, smart glasses or a tablet terminal are exemplified as the augmented reality device possessed by the second worker.

In Patent Document 3, in a camera control system configured to perform pan/tilt control of a camera for photographing an object to be monitored remotely, panning of the camera is interlocked with the progress of a series of preset work procedures.・It is described that the tilt control is automatically executed. That is, in Patent Document 3, it is possible to detect deviation from a predetermined work procedure process based on the output of the sensor.

JP 2019-166475 A Japanese Patent Application Laid-Open No. 2020-035280 JP 2015-204506 A

However, the conventional technology described in Patent Document 1 does not monitor the work in the field work in real time, but is configured to provide a procedure for so-called troubleshooting, and does not establish a single-person work system. do not have.

The conventional technology described in Patent Literature 2, for example, updates the work list while coordinating work statuses among workers, and does not monitor the work status of individual workers.

In Patent Document 3, by remotely controlling the panning and tilting of the camera, it is possible to obtain an accurate image and check the progress of the work procedure. Preparatory work associated with the original work becomes complicated. Further, Japanese Patent Laid-Open No. 2002-200002 detects deviations in work processing, and does not guide an appropriate work procedure before starting work.

INDUSTRIAL APPLICABILITY The present invention can confirm the work procedure of the next item before the actual work of the next item when performing a series of works in order, and can establish the work of a single-person work site. The object is to obtain a system, a work support control device, and a work support control program.

A work support system according to the present invention includes a photographing device for photographing a work image of the worker by moving a photographing range to follow the movement of the field of view of the worker, and the work image to be visually recognized by the worker. A display device that displays an image including a work image and a work image captured by the imaging device are compared with a reference image that is stored in advance according to the work procedure, and the presentation information is formulated based on the judgment result of the suitability of the work. and a display control unit that displays the presentation information formulated by the formulation unit on the display device.

According to the work support system of the present invention, the photographing device can follow the movement of the worker's field of view, move the photographing range, and photograph the worker's work image.

The formulating unit compares a display device that displays an image including the work image so that the worker can visually recognize the work image and a work image captured by a photographing device with a reference image that is stored in advance according to the work procedure, and determines the work. The information to be presented is formulated based on the judgment result of suitability. For example, an appropriate working image is stored in advance, and whether the images match or disagree is analyzed. For example, it analyzes whether or not the valve opening/closing lever has been moved from the first position to the second position in chronological order.

The display control unit displays the presentation information formulated by the formulation unit on the display device. Since the work image can be visually recognized by the worker, the presented information is naturally recognized by the worker. Therefore, the worker can adjust the work according to the presented information.

As a result, when a series of work is executed in sequence, the work procedure for the next item can be checked before the actual work for the next item, and the propriety of the work can be confirmed on the spot. work can be established.

In the work support system of the present invention, the display control section displays the presentation information in duplicate on the display device with respect to a work image at the time when the work of a predetermined number of work man-hours is completed.

The work image at the time when the work of the predetermined work man-hours is completed is displayed on the display unit, and the presentation information is displayed so as to overlap the work image, so that the worker can easily understand which work the presentation information is for. can be understood.

In the work support system of the present invention, the presentation information formulated by the formulation unit is work procedure information for the next process when the work is proper, and corrects the work when the work is improper. It is characterized by being advice information for doing.

If the work is proper, the work procedure information for the next process is presented, and if the work is improper, the advice information for correcting the work is presented, so that the worker does not hesitate and make mistakes. You can proceed with the work without omissions or omissions in the procedure.

In the work support system of the present invention, the photographing device and the display device are connected to each other, and the wearable device is attachable to the worker, and the formulating unit and the display control unit are connected to the wearable device. and a work support control device that transmits and receives the work image and the presentation information via a communication device.

By using a wearable device, there is no need for a hand to hold an imaging device and a display device, and it is possible to always capture an imaging range equivalent to the field of view of a worker. Work efficiency is improved compared to working while holding a communication terminal such as a terminal.

The work support system of the present invention is characterized in that the work support control device is a communication terminal device carried by the worker wearing the wearable device.

General-purpose wearable devices that are cheaper than industrial wearable devices can be applied.

In the work support system of the present invention, the photographing device and the display device are connected to each other, and the wearable device is wearable by the worker, and the wearable device includes the formulating unit and the display control unit. is incorporated, and the work support control device transmits and receives the work image and the presentation information via a communication device.

Although some improvements are required, general-purpose wearable devices that are inexpensive compared to industrial wearable devices can be applied.

In the work support system of the present invention, the work support control device collectively aggregates work images from a plurality of wearable devices, transmits presentation information to each wearable device, and records work procedures in each place as a history. A database is provided for storing information, and the history information is reflected in the formulation of the presentation information.

For example, in addition to installing a work support control device that aggregates work images from multiple wearable devices (installation of work support bases), a database that accumulates work procedures in each region as history information is installed in the work support control device. By doing so, the history information can be reflected in the formulation of the presentation information.

A work support control device according to the present invention moves a photographing range following movement of a worker's field of view, and stores work images photographed by a photographing device for photographing the work images of the worker in advance according to a work procedure. A formulating unit that compares against a reference image that is a reference that has been prepared and formulates presentation information based on the judgment result of work propriety, and a display control unit that displays the presentation information formulated by the formulating unit on a display device. have.

According to the work support control device of the present invention, the formulating unit moves the photographing range following the movement of the worker's field of view, and the work image photographed by the photographing device for photographing the worker's work image is The information to be presented is determined based on the judgment result of whether the work is appropriate or not by collating it with a reference image that is stored in advance according to the work procedure.

The display control unit displays the presentation information formulated by the formulation unit on the display device.

As a result, when a series of work is executed sequentially, the work procedure of the next item can be confirmed before the actual work of the next item, and the work at the work site of the one-man system can be established.

In the work support control device according to the present invention, the display control unit displays, on the display device, a work image at the time when work of a predetermined number of work man-hours is completed, and overlaps the presentation information on the work image. It is characterized by displaying as

The work image at the time when the work of the predetermined work man-hours is completed is displayed on the display unit, and the presentation information is displayed so as to overlap the work image, so that the worker can easily understand which work the presentation information is for. can be understood.

In the work support control device according to the present invention, the presentation information formulated by the formulation unit is work procedure information for the next process when the work is appropriate, and the work is performed when the work is inappropriate. It is characterized by being advice information for correction.

If the work is proper, the work procedure information for the next process is presented, and if the work is improper, advice information for correcting the work is presented.

Therefore, the operator does not get lost and does not rely on memory. Therefore, the work can be carried out without omissions or omissions in the procedure.

A work support control program according to the present invention is characterized by causing a computer to operate as each part of the above work support control device.

According to the work support program of the present invention, when a series of work is executed sequentially, the work procedure of the next item can be confirmed before the actual work of the next item, and the work of the work site of the one-person system is established. can do.

According to the present invention, when a series of work is executed sequentially, the work procedure of the next item can be confirmed before the actual work of the next item, and the work at the work site of the one-person system can be established. .

FIG. 10 is a work support system diagram for supporting a work procedure through cooperation between smart glasses and a smartphone according to the present embodiment. 1 is a perspective view of a worker wearing smart glasses and carrying a smartphone according to the present embodiment. FIG. 4 is a flowchart showing the flow of work support control according to the embodiment; (A) is a side view of a worker working, and (B) is a perspective view of an image corresponding to the field of view of the worker. (A) is a perspective view of smart glasses displaying a work image and a presentation image, and (B) is an enlarged view of the display image. FIG. 10 is a perspective view of a worker when an IC chip that executes work support function control is retrofitted to commercially available smart glasses according to Modification 1; FIG. 10 is a system diagram for remotely installing a work support control device and managing smart glasses via a network according to Modification 2;

FIG. 1 describes a work support system according to the present embodiment in which smart glasses 10 as a wearable communication terminal and a smartphone 12 as a communication terminal device are linked.

The smart glasses 10 are composed of an input device 14 , an output device 16 and a control section 18 . In this embodiment, the camera section 20 (see FIG. 2) is applied as the input device 14, and the monitor section 22 (see FIG. 2) is applied as the output device.

The control unit 18 also has an image control function for transmitting and receiving images, an audio control function for transmitting and receiving audio, and a communication control function for communicating image information and audio information with the smartphone 12 .

As shown in FIG. 2 , the smart glass 10 has a body portion 24 provided with a pair of openings, and a pair of frame portions 26 attached to left and right ends of the body portion 24 .

A pair of openings of the main body portion 24 are adapted to face the eyes of the operator 28 when worn, and the lens portion 30 is attached thereto.

A substantially L-shaped arm portion 32 protrudes forward of the operator 28 from the joint portion between the frame portion 26 and the lens portion 30 .

A camera section 20 is attached to the arm section 32 and is capable of photographing an image conforming to the line of sight of the worker 28 through the lens section 30 .

A monitor section 22 is attached to the arm section 32 to display an image within the field of view of the operator 28 through the lens section 30 .

In addition, the control section 18 is incorporated in the other frame section 26 . The control unit 18 transmits an image captured by the camera unit 20 to the smartphone 12 and displays an image received from the smartphone 12 on the monitor unit 22 . Note that the smart glasses 10 may be attached with a speaker (not shown) that outputs the sound received from the smartphone 12 .

The worker 28 can perform work support control by wearing the smart glasses 10 and carrying the smartphone 12 (for example, holding it in a pocket of work clothes).

In the present embodiment, the smartphone 12 functions as a communication terminal device capable of executing work support control, in addition to the functions generally defined as “a mobile phone having an operating system for mobile devices”. Regardless of the worker 28, carrying the smart phone 12 is a daily occurrence, and using the smart phone 12 as a communication terminal device capable of executing work support control is also useful from the standpoint of convenience.

FIG. 1 shows a functional block diagram of a smartphone 12 that operates as a communication terminal device capable of executing work support control. Note that each functional block may be incorporated into the smartphone 12 as a hardware configuration of an IC chip (such as an ASIC), or part or all of it may be implemented as a work support control program (software) that can be executed by the smartphone 12. May be incorporated.

As shown in FIG. 1 , the work support control function in the smart phone 12 has a communication I/F 50 . The communication I/F 50 is capable of using functions inherent in the smartphone 12 (for example, Bluetooth (registered trademark)).

The communication I/F 50 is connected to the image receiving unit 52 and receives images (captured image information) captured by the camera unit 20 mounted on the smart glasses 10 .

The image reception unit 52 is connected to the image analysis unit 54 , and the received photographed image information is sent to the image analysis unit 54 . The image analysis unit 54 extracts images at the start and end of a predetermined division of work processes (N (N is a positive integer) types of unit work procedures).

The image analysis section 54 is connected to the synchronization signal generation section 56 .

The synchronizing signal generation unit 56 sends the photographed image information to the work procedure collation unit 58, sets a time table (timer starts) from the time when the delimitation of the work process is extracted, and stores the time table in the work procedure database 60 in advance. Instruct to search for standard operating procedure images.

The reference work procedure image retrieved from the work procedure database 60 is sent to the work procedure matching unit 58 .

The work procedure matching unit 58 compares the start image and the end image of the Nth work procedure. The result of collation by the work procedure collating section 58 is sent to the work procedure adequacy determining section 62 .

The work procedure suitability determination unit 62 determines whether or not the end image is matched within a predetermined time after the start visible image of the Nth work procedure is matched. That is, there are two determination results as follows.

(Adequacy Judgment) After the start visible image of the Nth work procedure was collated, the end image was collated within a predetermined time.

(Determination of Inappropriateness) After the start visible image of the N-th work procedure was collated, the end image was not collated within a predetermined time.

The work procedure adequacy determination unit 62 is connected to the provided information reading unit 64 and sends one of the determination results to the provided information reading unit 64 .

In the provided information reading unit 64, information to be notified to the worker 28 (image information to be displayed on the monitor unit 22 of the smart glasses 10) is read from the work procedure database 60 according to the determination result, and is read through the communication I / F 50. , to the smartphone 12 .

The provided information reading unit 64 reads information about the work procedure of the next process when the determination result is "determined as appropriate", and reads information about the work procedure of the current process when the determination result is "determined as inappropriate". 10. In addition, in the case of "inappropriate judgment", it is possible to instruct to redo or present a work procedure more detailed than usual. In addition to reading the provided information, for example, a skilled person or the like waiting at the management center may be contacted.

The operation of this embodiment will be described below.

FIG. 3 is a flow chart showing the flow of work support control processing according to the present embodiment. When the work support control is activated, an image (through image) captured by the camera unit 20 is displayed on the monitor unit 22 of the smart glasses 10 .

At step 100 , the type of work to be executed and the work procedure are specified, and the process proceeds to step 102 . At step 102, the variable N indicating the work procedure is reset (N=0).

At the next step 104 , the variable N is incremented (N←N+1) and the process proceeds to step 106 .

In step 106, the work information of the Nth work procedure is read out, the process proceeds to step 108, the work information is superimposed on the captured image displayed on the monitor unit 22 of the smart glasses 10, and the process proceeds to step 110. .

At step 110 , acquisition of captured images from the smart glasses 10 is started.

In the next step 112, a work start trigger image for the N-th work procedure is extracted by image analysis, and the process proceeds to step 114 to generate a synchronizing signal (timer start). The work start trigger image may be extracted by any of manual operation by the worker 28, reading of identification code information, and image analysis.

In the next step 116 , the photographed image of the actual work is compared with the reference image stored in the work procedure database 60 , and the process proceeds to step 118 . The comparison between the photographed image and the reference image may be performed by comparing a series of moving images at predetermined intervals (monitoring during work), but at least a still image taken at the site at the end of the Nth work procedure The image may be collated with a reference image (still image) at the end of the work. If there is monitoring during the work, it becomes easier to give appropriate instructions such as instructions to redo the work.

At step 118, it is determined whether or not the work end trigger image for the Nth work procedure has been extracted. If a negative determination is made in step 120, the process returns to step 116, and steps 116, 118 and 120 are repeated until an affirmative determination is made in step 118 or step 120.

Here, if an affirmative determination is made in step 120, it is determined that the end trigger of the N-th work procedure could not be extracted within the predetermined time, and the N-th work was judged to be inappropriate. Provided information indicating that the operation of the procedure is inappropriate, and proceeds to step 124 .

At step 124, a troubleshooting process is performed. Troubleshooting includes, for example, instructing to redo work, presenting detailed work procedures, and contacting experts.

In the next step 126, the work procedure N to be continued is entered, and the process returns to step 106. At step 126, the following work procedure N can be set.

(1) Input the current variable N to repeat the inappropriate work under the work support control.

(2) Remove the inappropriate work procedure from the monitoring target, for example, to ensure that the work is completed under the guidance of an expert, and to continue the work support control from the next work process, the current variable Enter N+1 as N.

On the other hand, if an affirmative determination is made in step 118, the end trigger image of the N-th work procedure can be extracted within a predetermined time, it is determined that the N-th work has been properly completed, the process proceeds to step 128, and the N-th work procedure is performed. is notified that the work has been properly completed, and the process proceeds to step 130 .

At step 130, it is determined whether or not all the work has been completed. If the determination is negative, the process returns to step 104, N is incremented, and the above steps are repeated.

Also, if the determination at step 130 is affirmative, this routine ends.

FIGS. 4 and 5 show an example of transition of work performed under the work support system according to the present embodiment in which the smart glasses 10 and the smartphone 12 are linked.

As shown in FIG. 4A , a worker 28 facing a work site 66 wears smart glasses 10 and carries a smartphone 12 .

As shown in FIG. 4B, since the worker 28 is looking at the work site, the camera unit 20 of the smart glasses 10 captures the visual field of the worker 28, that is, the image shown in FIG. will be photographed.

When the work continues and the Nth work is started, as shown in FIG. Information is superimposed.

The worker 28 sees an image displayed on the monitor unit 22 in the visual field range that he or she is viewing, which is equivalent to the image that the worker is directly viewing, and provides the provided information (here, in the Nth work procedure As shown in FIG. 5B, a character message "Please open the gas valve in the direction of the arrow") is displayed.

As described above, in the present embodiment, the commercially available smart glasses 10 are applied as a wearable communication terminal of the work support control system, and the work support control program, which is a feature of the present embodiment, is applied to the smartphone 12. By installing it as an application program, communication is performed between the smartphone 12 and the smart glasses 10 to perform work support control.

For this reason, for example, there is no need to develop or purchase (or lease) smart glasses with special functions for limited work procedures. Support control is possible, and the operator 28 can eliminate the complication of working with both hands while holding the work procedure manual or the like with one hand.

(Modification 1)

Modification 1 of the present embodiment will be described below.

In the present embodiment, work support control is realized by cooperation between the commercially available smart glasses 10 and the smartphone 12, but in Modification 1, as shown in FIG. , a housing 68 containing an IC chip functioning as the communication terminal device of the present invention is attached as a so-called post-installation.

In FIG. 6, the housing 68 is attached to the outside of the frame portion 26, but the frame portion 26 may contain an IC chip.

Communication between the control unit 18 of the smart glasses 10 and the IC chip may be wireless or wired.

According to Modification 1, by simply remodeling the smart glasses 10, a system capable of performing work support control can be constructed with the smart glasses 10 alone.

(Modification 2)

Modification 2 of the present embodiment will be described below.

In the present embodiment, work support control is realized by cooperation between the commercially available smart glasses 10 and the smartphone 12, but in the second modification, as shown in FIG. The point is that communication with the center 70 is made possible via the network 72 .

As shown in FIG. 7 , the communication control function of the control unit 18 of the smart glasses 10 can communicate with the wireless communication device 72A of the network 72 . A work support control device 12A installed in the work support management center 70 is connected to the network 72 . The work support control device 12A has the same function as the communication terminal device (see FIG. 1) described as the function of the smart phone 12 described in the present embodiment, so detailed description of the configuration will be omitted.

In Modified Example 2, a user interface 74 is connected to the work support control device 10 . The user interface 74 has an output device (monitor section 74M) and an input device 74A (keyboard, microphone, etc.).

Here, in the embodiment described above, images at the start and end of work are extracted for each subdivided work procedure, and it is automatically determined whether or not each work procedure is properly performed.

In addition to the automatic determination control, the operator 76 monitors the image (work progress image) displayed on the monitor unit 22 to monitor whether or not the work of each work procedure is being performed in chronological order according to the standard work procedure. It becomes possible to

In addition, if the operator 76 is familiar with the work to be performed while wearing the monitor smart glasses 10, the operator 76 can quickly give work guidance when the worker 28 performs an inappropriate work.

Further, in the work support control device 12A, the work procedures can be accumulated as history information, and the history information can be reflected in the formulation of the presentation information. Note that this function can also be applied to the present embodiment (a form in which the smartphone 12 is applied as the work support control device). That is, even in the smartphone 12 shown in FIG. 1, the work procedure may be accumulated as history information and reflected in the formulation of the history information W and the presentation information.

10 smart glasses 12 smartphone 14 input device 16 output device 18 control unit (display control unit)
20 Camera section (shooting device)
22 Monitor (display device)
24 main body part 26 frame part 28 operator 30 lens part 32 arm part 50 communication I/F
52 Image reception unit 54 Image analysis unit 56 Synchronization signal generation unit 58 Work procedure matching unit 60 Work procedure database 62 Work procedure suitability determination unit (Formulation unit)
64 Provided information reading unit (formulation unit)
66 work site 68 housing 70 work support management center 72A wireless communication device 74 user interface 74M monitor unit 74A input device 76 operator

Claims (11)

a photographing device that follows the movement of the field of view of the worker and moves the photographing range to photograph the work image of the worker;
a display device that displays an image including the work image so that the worker can see it;
a formulating unit that compares the work image captured by the photographing device with a reference image that is stored in advance according to the work procedure and formulates presentation information based on the result of determination of the suitability of the work;
a display control unit that displays the presentation information formulated by the formulation unit on the display device;
work support system. 2. The work support system according to claim 1, wherein said display control unit displays said presentation information overlappingly on said display device with respect to a work image at the time when work of a predetermined number of work man-hours is completed. The presentation information formulated by the formulation unit is work procedure information for the next process when the work is proper, and is advice information for correcting the work when the work is improper. The work support system according to claim 1 or 2. A wearable device in which the photographing device and the display device are connected and which can be worn by the worker,
A work support control device in which the formulation unit and the display control unit are provided separately from the wearable device, and the work image and the presentation information are transmitted and received via a communication device,
The work support system according to any one of claims 1 to 3. 5. The work support system according to claim 4, wherein said work support control device is a communication terminal device possessed by a worker wearing said wearable device. A wearable device in which the photographing device and the display device are connected and which can be worn by the worker,
A work support control device in which the formulation unit and the display control unit are incorporated in the wearable device, and the work image and the presentation information are transmitted and received via a communication device,
The work support system according to any one of claims 1 to 3. The work support control device collectively aggregates work images from a plurality of wearable devices, transmits presentation information to each wearable device, and includes a database that accumulates work procedures in various places as history information, reflecting the history information in formulating the presentation information;
The work support system according to claim 4. A work image captured by a photographing device that captures a work image of the worker by moving the photographing range in accordance with the movement of the worker's field of view is collated with a reference image that is stored in advance according to the work procedure. , a formulation department that formulates presentation information based on the judgment result of work propriety;
a display control unit that displays the presentation information formulated by the formulation unit on a display device;
A work support control device having 9. The display control unit according to claim 8, wherein the display control unit displays, on the display device, a work image at the time when work of a predetermined work man-hour is completed, and displays the presentation information so as to overlap the work image. Work support controller. The presentation information formulated by the formulation unit is work procedure information for the next process when the work is proper, and is advice information for correcting the work when the work is improper. The work support control device according to claim 8 or 9. Operating a computer as each part of the work support control device according to any one of claims 8 to 10,
Work support control program.

Images