JP2021076990A - Work display method and work display apparatus - Google Patents

Abstract

To provide a work display method which can reduce detection errors while reducing a processing load in determination processing.SOLUTION: A work display method includes: first display step for displaying first work in which a first component is mounted in a first area and the first component is not mounted in a second area; and second display step for displaying second work in which a second component is mounted in the second area and the second component is not mounted in the first area. The first display step validates detecting the first component in the first area and invalidates detecting the second component in the second area (S34), and outputs information indicating that the first work has been normally conducted (S37) when the first component is detected in the first area (Yes in S36). The second display step validates detecting the first component in the second area and invalidates detecting the second component in the first area (S34), and outputs information indicating that the second work has been normally conducted (S37) when the second component is detected in the second area (Yes in S36).SELECTED DRAWING: Figure 6

Description

The present invention relates to a work display method and a work display device.

Conventionally, it has been studied to detect an assembly error of a worker who performs an assembly work in a factory or the like. According to Patent Document 1, whether or not a worker has collected a part from a correct part mounting portion among a plurality of component mounting portions by detecting the position of the worker's hand based on an image taken by a camera. A production system (work display system) for determining whether or not a product is disclosed.

Japanese Unexamined Patent Publication No. 2018-45669

By the way, in such a production system, it is desired that the processing load in the determination process is reduced and erroneous detection is suppressed.

Therefore, the present invention provides a work display method and a work display device capable of suppressing erroneous detection while reducing the processing load in the determination process.

The work display method according to one aspect of the present invention is the first work in which the first component is mounted in the first region and the first component is not mounted in the second region different from the first region. A work content is provided to a worker who performs a plurality of operations including a second operation in which the second component is mounted in the second area and the second component is not mounted in the first area. A work display method for displaying, the first display step for displaying the first work content in the first work, and the second display step for displaying the second work content in the second work. In the first display step, the detection of the first component in the first region is enabled, and the detection of the second component in the second region is invalidated, and the sensing result obtained from the sensor unit is included. When the first component is detected in the first region based on the above, information indicating that the first work has been normally performed is output to the display device as the first work content, and the second display is performed. In the step, the detection of the second component in the second region is enabled, the detection of the first component in the first region is invalidated, and the second component is obtained based on the sensing result acquired from the sensor unit. When the second component is detected in the area, information indicating that the second work has been normally performed is output to the display device as the content of the second work.

Further, the work display device according to one aspect of the present invention is a work in which the first component is mounted in the first region and the first component is not mounted in the second region different from the first region. Work for a worker who performs a plurality of operations including the work and the second work in which the second component is mounted in the second area and the second component is not mounted in the first area. It is a work display device for displaying the contents, and when the first work is performed, the detection of the first component in the first region is enabled, and the second component in the second region is enabled. Control that invalidates the detection and enables the detection of the second component in the second region and invalidates the detection of the first component in the first region when the second operation is performed. When the first part is detected in the first region based on the sensing result acquired from the sensor unit when the unit and the first operation are performed, the first operation is normally performed. When the information indicating that is output to the display device as the first work content and the second work is performed, the second component is displayed in the second region based on the sensing result acquired from the sensor unit. When detected, it includes an output unit that outputs information indicating that the second work has been normally performed to the display device as the content of the second work.

It should be noted that these comprehensive or specific embodiments may be realized in a recording medium such as a system, method, integrated circuit, computer program or computer-readable CD-ROM, and the system, integrated circuit, computer program and recording. It may be realized by any combination of media.

According to the work display method or the like according to one aspect of the present invention, erroneous detection can be suppressed while reducing the processing load in the determination process.

FIG. 1 is a block diagram showing a functional configuration of a work instruction system according to an embodiment. FIG. 2A is a diagram showing an example of a working table. FIG. 2B is a diagram showing an example of a detection table. FIG. 3 is a schematic diagram showing a detection region according to the embodiment. FIG. 4A is a diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 4B is a second diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 4C is a third diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 4D is a diagram for explaining the basic operation of the work instruction device according to the embodiment. FIG. 5 is a sequence diagram showing the operation of the work instruction system according to the embodiment. FIG. 6 is a flowchart showing the operation of the work instruction device according to the embodiment. FIG. 7A is a diagram showing how the detection of the first component in the first region is effective. FIG. 7B is a diagram showing how the detection of the second component in the second region is effective. FIG. 8 is a flowchart showing the operation of the work instruction device according to the first modification of the embodiment. FIG. 9 is a diagram showing how the prohibited area corresponding to the work 1 is effective. FIG. 10 is a flowchart showing the operation of the work instruction device according to the second modification of the embodiment.

Hereinafter, embodiments will be described with reference to the drawings. Each of the embodiments described below provides a comprehensive or specific example. The numerical values, shapes, materials, components, arrangement positions and connection forms of the components, steps (steps), order of steps (steps), etc. shown in the following embodiments are examples, and the gist of limiting the present invention. is not it. Further, among the components in the following embodiments, the components not described in the independent claims will be described as arbitrary components.

Further, each figure is a schematic view and is not necessarily exactly illustrated. Further, in each figure, substantially the same configuration may be designated by the same reference numerals, and duplicate description may be omitted or simplified.

(Embodiment)
[1. Work instruction system configuration]
First, the configuration of the work instruction system according to the embodiment will be described with reference to FIGS. 1 to 3. FIG. 1 is a block diagram showing a functional configuration of the work instruction system 1 according to the embodiment.

As shown in FIG. 1, the work instruction system 1 includes a work instruction device 10, a first driver 20, a second driver 30, an image pickup device 40, and a display device 70. In the following, an example in which the work instruction system 1 includes two tools (first driver 20 and second driver 30) will be described, but the present invention is not limited thereto. In the following, when the first driver 20 and the second driver 30 are not distinguished, they are also simply described as drivers. The work instruction system 1 is an example of a work display system.

The work instruction device 10 is a server device for preventing work mistakes, supporting proficiency, observing work time, improving work, and notifying work mistakes. The basic operation of the work instruction device 10 and the like will be described later. The work instruction device 10 according to the present embodiment, for example, detects work instructions and work errors for a worker who performs a single assembly work in which one person is in charge of a plurality of operations in the assembly work and the like. The work performed by the operator includes, for example, a work using a tool (for example, a screwdriver) and a work not using a tool. A tool-free operation is an operation that does not use all of the tools used in any of the plurality of operations. The work without tools includes a manual work, for example, a work of inserting a washer, a work of tightening a clip, and a work of inspection. The work instruction device 10 is an example of a work display device for displaying the work contents in the work performed by the worker on the display device 70.

The work instruction device 10 has a communication unit 11, a control unit 12, and a storage unit 13.

The communication unit 11 is a communication interface that is communicably connected to various devices included in the work instruction system 1 via wireless communication or wired communication. The communication unit 11 outputs, for example, control information for controlling the image pickup unit 60 generated by the control unit 12 to the image pickup apparatus 40. Further, the communication unit 11 acquires information based on the image data captured by the image pickup unit 60 from the image pickup apparatus 40. In this way, the communication unit 11 functions as an output unit that outputs control information and an acquisition unit that acquires information based on image data. The image data is an example of the sensing result (sensing data).

Further, the communication unit 11 may acquire a start signal indicating the start of each of the plurality of operations and an end signal indicating the end. For example, when the work is a work using a driver, the communication unit 11 may acquire at least one of a start signal and an end signal from the first driver 20 or the second driver 30. Further, the communication unit 11 may acquire at least one of the start signal and the end signal by an operator's operation (for example, an operation on a button and a foot switch (not shown)).

The control unit 12 generates and outputs information on the processing executed by the device to various devices connected to the work instruction device 10, and the device acquires the information from the various devices (for example, acquired by measuring). It is a processing unit that acquires information and performs predetermined processing based on the information.

Specifically, the control unit 12 obtains driver control information for controlling the first driver 20 and the second driver 30 based on, for example, the work table T1 (see FIG. 2A) stored in the storage unit 13. Output to the 1st driver 20 and the 2nd driver 30. Further, the control unit 12 controls each camera (for example, cameras a to d) of the image pickup apparatus 40 based on the detection table T2 (see FIG. 2B) stored in the storage unit 13. The information is output to the image pickup apparatus 40.

Further, the control unit 12 determines whether or not the work has been normally performed based on the information based on the image data of each camera acquired from the image pickup apparatus 40. Then, the control unit 12 outputs information for displaying the determination result to the display device 70 via the communication unit 11.

The storage unit 13 is a storage device that stores a control program executed by the control unit 12 included in the work instruction device 10. Further, the storage unit 13 stores the work table T1 and the detection table T2. Further, the storage unit 13 may store information acquired via the communication unit 11 (for example, information based on image data). The storage unit 13 is realized by, for example, a semiconductor memory or the like.

Here, the work table T1 and the detection table T2 stored in the storage unit 13 will be described with reference to FIGS. 2A and 2B. FIG. 2A is a diagram showing an example of the work table T1. Note that FIG. 2A shows a case where a screw is tightened as an example of a screw. In the following, screws are also simply referred to as screws.

As shown in FIG. 2A, the storage unit 13 stores the work table T1 in which the work procedure, the work name, the driver name, the number of screws, and the upper and lower limit pulses are associated with each other. The work of the work procedure 1, 2 and 4 is an example of the work using the driver, and the work of the work procedure 3 is an example of the work not using the driver. In other words, the work table T1 is a table for performing a plurality of operations including a work using a driver and a work not using a driver. The number and order of the work using the driver and the work not using the driver included in the work table T1 are not particularly limited.

The work procedure indicates the order of a plurality of works performed by the worker. The work procedure is predetermined. In the example of FIG. 2A, it is shown that the work procedures 1 to 4 are performed in that order.

The work name indicates the work content and is set for each of the plurality of works. The worker can know the contents of the work just by looking at the work name.

The driver name indicates a name for the operator to identify the driver, and is set for each driver. The driver name may be the model number of the driver or the like as long as the operator can identify the driver.

The number of screws indicates the number of screws to be tightened in the work.

The upper limit pulse and the lower limit pulse are threshold values (number of pulses) used by the control unit 12 to determine whether or not the work using the driver is completed based on the tightening information of the screw. The upper limit pulse and the lower limit pulse are determined, for example, based on the number of rotations (or the number of pulses) when the screw tightening in the screw tightening work (screw tightening work) is properly completed by each of the plurality of workers. The upper limit pulse and the lower limit pulse are based on, for example, a rotation speed of a predetermined number or more (for example, a maximum rotation speed) and a rotation speed of a predetermined value or less (for example, a minimum rotation speed) among the rotation speeds of each of a plurality of workers. It may be decided. As a result, the upper limit pulse and the lower limit pulse can be appropriately set even if the rotation speeds when the screw tightening in the screw tightening work is properly completed are different for each of the plurality of workers.

The working table T1 may include at least one of an upper limit pulse and a lower limit pulse. Further, when the screw tightening information includes torque information, the work table T1 may include at least one of an upper limit and a lower limit of torque. Further, the work table T1 does not have to include the information of the upper limit pulse and the lower limit pulse when the driver determines whether or not the work is completed.

The work of the work procedure 1 (hereinafter, also referred to as the work 1) indicates that the work is a work of tightening one screw A by using the driver X. The driver X is, for example, the first driver 20. Further, the upper limit pulse and the lower limit pulse are criteria for determining whether or not the screw tightening of the screw A is completed when the screw A is screwed using the driver X, and correspond to the rotation speed of the screw A. It is a numerical value to be performed.

The work of the work procedure 2 (hereinafter, also referred to as the work 2) indicates that the work is a work of tightening one screw B by using the driver Y. In work 2, a driver different from that in work 1 is used. The driver Y is, for example, the second driver 30.

The work of the work procedure 3 (hereinafter, also referred to as the work 3) shows an example in which one plate C is attached.

The work of the work procedure 4 (hereinafter, also referred to as the work 4) shows an example in which one screw D different from the screw A is tightened by using the same driver as the work procedure 1.

As described above, the work instruction device 10 provides information for an operator to perform a plurality of operations including a work using a driver (for example, a screw tightening work) and a work not using a driver according to a predetermined work procedure. (For example, the work table T1) is held.

Next, the detection table T2 stored in the storage unit 13 will be described. FIG. 2B is a diagram showing an example of the detection table T2.

As shown in FIG. 2B, the storage unit 13 stores the detection table T2 in which the work procedure, the work name, and the information about the image pickup apparatus 40 are associated with each other. The information about the image pickup apparatus 40 includes a detection camera, a prohibited camera, a stop camera, and a detection color. The information regarding the image pickup apparatus 40 may include at least information regarding the detection camera.

The detection camera refers to a camera that is operated to determine whether or not the current work has been performed normally among a plurality of cameras. For example, in work 1, an example in which the camera a among the cameras a to d is operated as a detection camera is shown.

The prohibited camera indicates a camera that is operated to determine whether or not a component in the work is mounted in a prohibited area in which the component should not be mounted in the current work among a plurality of cameras. For example, in work 1, an example of operating the cameras b and d among the cameras a to d as prohibited cameras is shown. Further, for example, in operations 3 and 4, an example in which none of the cameras is operated as a prohibited camera is shown.

The stop camera indicates a camera that stops operation in the current work among a plurality of cameras. That is, the camera described in the stop camera does not shoot in the work. For example, in work 1, an example of stopping the operation of the camera c among the cameras a to d is shown. At least one camera is set as the stop camera.

The detection color indicates the color of the part for determining whether or not the work has been performed normally. For example, in the work 1, it is shown that the color of the part needs to be "blue" in order for the work 1 to be performed normally.

With reference to FIG. 1 again, the first driver 20 and the second driver 30 are examples of power-operated tools used by the operator in the tool-using work. The tool here means, for example, a power tool. Further, the power is electricity, air, or the like. The first driver 20 is, for example, a fastening tool used for screw tightening of screws A and D, and is an example of the first tool. The second driver 30 is a fastening tool different from that of the first driver 20. The second driver 30 is, for example, a fastening tool used for screw tightening of a screw B, and is an example of a second tool. The number and types of drivers connected to the work instruction device 10 are not particularly limited.

Although not shown, each of the first driver 20 and the second driver 30 has a communication unit for communicating with the work instruction device 10, a pulse measurement unit for measuring the rotation speed of the driver, torque increase, rotation start, and rotation. It has a detection unit that detects the end of rotation. The pulse measuring unit has, for example, a circuit incorporated in a driver motor (not shown) and capable of measuring a pulse signal emitted by an encoder (not shown), and detects the rotation speed from the measurement result of the pulse signal. When the motor rotates, the encoder emits a pulse signal as a rotation signal corresponding to the rotation. The detection unit detects torque increase, rotation start and rotation end, for example, by measuring the current consumption of the driver. The communication unit operates at least one of the measurement result measured by the pulse measurement unit and the detection result detected by the detection unit (for example, the detection result of the rotation start and the rotation end, which is an example of the start signal and the end signal). Output to the indicator device 10.

Each of the first driver 20 and the second driver 30 may have a processing unit for determining whether or not screw tightening is completed based on the rotation speed measured by the pulse measuring unit.

The image pickup device 40 takes a picture of the detection area based on the control information from the work instruction device 10, and outputs image data indicating the taken image to the work instruction device 10. The image pickup device 40 includes a camera control unit 50 and an image pickup unit 60. The detection area is an area in the main body 100 (see FIG. 3) including a position where a component is attached. The main body 100 may be a substrate or the like on which components are mounted.

The camera control unit 50 is a processing unit that controls the operation of the image pickup unit 60 based on the control information from the work instruction device 10. Further, the camera control unit 50 acquires image data from the image pickup unit 60 and outputs information based on the acquired image data to the work instruction device 10.

The information based on the image data may include the image data itself, or may include an analysis result (for example, an image analysis result) obtained by analyzing the image data.

The camera control unit 50 is realized by, for example, a microcomputer. The camera control unit 50 reads a control program from the built-in storage unit and executes the read program.

The imaging unit 60 has one or more cameras that take pictures to detect whether or not the component is mounted at a predetermined position. In the present embodiment, the imaging unit 60 is composed of cameras a to d, but the number of cameras is not particularly limited. Each of the cameras a to d shoots a detection area set for each camera, and outputs the image data acquired by the shooting to the camera control unit 50. Each of the cameras a to d may be cameras having the same performance (for example, resolution) or cameras having different performances from each other. The cameras a to d may be appropriately determined according to, for example, the work content, the detection accuracy, and the like. The image pickup unit 60 is an example of a sensor unit.

The work instruction system 1 may include a sensor device other than the image pickup device 40 as long as the component can be detected. The work instruction system 1 may include, for example, a sensor device other than the image pickup device 40 in place of the image pickup device 40 or together with the image pickup device 40. The sensor included in the sensor device is not particularly limited, but may include, for example, an optical sensor that detects the presence or absence of a component by shading.

The image pickup device 40 does not have to have the camera control unit 50. In this case, the work instruction device 10 may have, for example, the function of the camera control unit 50. The work instruction device 10 may output control information to each camera of the image pickup unit 60 via the communication unit 11 and acquire image data from each camera of the image pickup unit 60.

As shown in FIG. 2B, in the present embodiment, the camera a is a camera for detecting whether or not there is a screw A corresponding to the work 1 in the detection area in the work 1. The camera b is a camera for detecting whether or not there is a screw B corresponding to the work 2 in the detection area in the work 2. The camera c is a camera for detecting whether or not there is a plate C corresponding to the work 3 in the detection area in the work 3. The camera d is a camera for detecting whether or not there is a screw D corresponding to the work 4 in the detection area in the work 4.

The work 1 is a work (an example of the first work) in which the first component (for example, screw A) is mounted in the first region and the first component is not mounted in the second region different from the first region. The work 2 is a work in which the second component (for example, screw B) is mounted in the second region and the second component is not mounted in the first region (an example of the second work). Further, in the first work, it is prohibited to mount the first component in the second and fourth regions, and in the second work, it is prohibited to mount the second component in the fourth region. ing. Therefore, in the first work, the cameras b and d are set in the prohibited camera, and in the second work, the camera d is set in the prohibited camera.

Here, the detection region of the imaging unit 60 will be described with reference to FIG. FIG. 3 is a schematic diagram showing detection regions Ra to Rd according to the embodiment. FIG. 3 shows a main body 100 to which screws A, screws B, a plate C, and a screw D are attached (attached), and a detection area corresponding to each component. The plate C is arranged so as to cover the screw A. The region surrounded by the alternate long and short dash line shown in FIG. 3 indicates the detection region.

As shown in FIG. 3, the detection region Ra is a region of the main body 100 to which the screw A is mounted. The detection area Ra is photographed by the camera a. The detection area Rb is an area of the main body 100 to which the screw B is mounted. The detection area Rb is photographed by the camera b. The detection area Rc is an area of the main body 100 to which the plate C is attached. The detection area Rc is photographed by the camera c. The detection area Rd is an area of the main body 100 to which the screw D is mounted. The detection area Rd is photographed by the camera d. At least a part of the detection regions Ra to Rd may overlap in a plan view. In the present embodiment, the detection region Rc is a region including the detection region Ra in a plan view. Each of the detection regions Ra to Rd may be, for example, a region that does not overlap with each other.

Further, the detection areas Rb and Rd are prohibited areas corresponding to the work 1, and are photographed by the cameras b and d in the work 1, respectively. The detection area Rd is a prohibited area corresponding to the work 2, and is photographed by the camera d in the work 2.

The prohibited area is, for example, an area in which parts used in a certain work must not be mounted. The prohibited area may be, for example, an area in which a part having a similar appearance to a part used in a certain work is mounted. As described above, the prohibited area is an area for detecting that the operator attaches the component to the wrong area in the work.

When the sensor device is an image pickup device that shoots at the angle of view shown in FIG. 3, that is, when the image pickup device can shoot an image including all of a plurality of detection regions Ra to Rd, the image pickup unit 60 It may be composed of one camera.

With reference to FIG. 1 again, the display device 70 displays the information acquired from the work instruction device 10 as an image. Images include photographs, videos, illustrations, text and the like. The display device 70 is realized by a liquid crystal display or the like. The image output by the display device 70 is visually recognized by the operator and used for confirming the work content and the determination result. As a determination result, the display device 70 indicates that the work currently being performed has been performed normally, information indicating that the work has not been performed normally, and that the work has not been performed. Display the information to be shown.

In addition, the display device 70 displays information about the work currently being performed. The display device 70 displays, for example, at least one of a work procedure, information specifying a driver to be used (for example, a driver name), the number of screws to be tightened, a determination result of screw tightening, and an elapsed work time. The display device 70 is arranged at a position where it does not interfere with the work of the operator.

The display device 70 is an example of an output device that outputs information acquired from the work instruction device 10. As an output device, the work instruction system 1 is a device that displays information on an object (for example, a screen) such as a sound output device (for example, a speaker) or a projector, in place of the display device 70 or together with the display device 70. It may have a device such as a light emitting device that outputs information with light (for example, the color of light).

[2. Basic operation of work instruction system]
Next, the basic operation in the work instruction system 1 as described above will be described with reference to FIGS. 4A to 4C. Specifically, the operation in which the work instruction device 10 gives a work instruction to the operator will be described. This is realized by each processing unit of the work instruction device 10 executing a predetermined application program (hereinafter, also referred to as a dedicated application).

First, a basic operation performed using a work instruction image or the like stored in the storage unit 13 will be described. FIG. 4A is a diagram for explaining the basic operation of the work instruction device 10 according to the embodiment.

As shown in FIG. 4A, the operator first starts the dedicated application and inputs predetermined information in the items displayed in the menu. Predetermined information includes, for example, information for identifying a worker and a work performed by the worker. The items displayed in the menu are, for example, "worker input" for inputting information that identifies the worker, "terminal No. input" for specifying the work terminal to be used, and "input of the product number of the product to be worked on". Enter the product number. " In addition, the items displayed in the menu include handling of actual data acquired when working in the past, adjustment of screen layout, and the like.

When the predetermined information is input, the display device 70 displays an image corresponding to the work. FIG. 4B is a second diagram for explaining the basic operation of the work instruction device 10 according to the embodiment. Specifically, the work instruction image P displayed on the display device 70 by the work instruction device 10 is shown.

As shown in FIG. 4B, when the product type is set (“DEMO-2017” is set in FIG. 4B), the work instruction image P is displayed on the display device 70. When the work instruction device 10 acquires an operation from the operator, the work instruction image P corresponding to the operation is displayed on the display device 70. The work instruction image P includes the product type, the work name p1 (in FIG. 4B, "take part A"), precautions p2 during work (in FIG. 4B, "take surely"), and an explanation explaining the work. The image p3, the part name p4 (“part A” in FIG. 4B), and the like are displayed. Further, although not shown, the work instruction system 1 is provided with a sound output device, and a work instruction corresponding to the explanatory image p3 is output by voice from the sound output device. The explanatory image p3 is, for example, a moving image. Further, the work name p1 (work content), precautions p2, explanatory image p3, part name p4, and the like are created for each of the plurality of works.

As a result, the worker receives the work instruction by the work instruction image P and the voice. The work instruction device 10 can support the prevention of work mistakes by the operator by displaying the work instruction by the explanatory image p3 and the voice, the work instruction by the character, the caution p2, and the like. Further, for example, when the worker is an expert, the work can be performed by voice instruction by the sound output device without looking at the explanatory image p3 of the display device 70. Further, for example, when the worker is not accustomed to the work, the work can be smoothly proceeded by viewing the explanatory image p3 of the display device 70 only when necessary. Therefore, according to the work instruction device 10, it is possible to prevent work mistakes and support proficiency.

For example, when the work is a work using a driver, the work name p1 includes the driver to be used, the type of screw, and the number of screws, and is displayed, for example, "Tighten screw 1 in two places with driver X". To.

Further, the work instruction image P may further display a selection image for selecting the worker classification according to the skill level of the worker. FIG. 4C is a third diagram for explaining the basic operation of the work instruction device 10 according to the embodiment.

As shown in FIG. 4C, when the worker clicks the drop-down menu or the like before or during the work, the selection image p5 for selecting the worker classification of the worker is displayed. In the example of FIG. 4C, an example in which three categories of "newcomer", "normal", and "skilled" are displayed is shown, but the number of worker categories is not particularly limited and may be 2 or more. .. The output content on the display device 70 may be changed by the worker selecting his / her own worker category. For example, when the "newcomer" is selected, the work instruction device 10 outputs all the work instructions stored in the storage unit 13. Further, when "normal" or "skilled" is selected, the work instruction device 10 does not have to output the work instruction of the preset work among the plurality of works. As a result, the work instruction system 1 can output a work instruction image P suitable for the skill level of the operator.

With reference to FIG. 4B again, the elapsed time display bar p6 is displayed on the work instruction image P. In the elapsed time display bar p6, a bar showing the progress rate of a plurality of works and the progress time are displayed. The bar showing the progress rate shows the elapsed time of the work currently being performed (“work unit progress” shown in the upper part of the elapsed time display bar p6), and the cumulative standard work time of each of the multiple works (elapsed time display bar p6). The “cumulative work base” shown in the middle row and the elapsed time during one cycle operation (“total work progress” shown in the lower row of the elapsed time display bar p6) are included. The standard working time is set in advance. It should be noted that a plurality of operations performed by the operator are also described as elemental operations. "One-cycle operation" means that the worker is in the process of performing a plurality of tasks according to the work procedure, assuming that the worker performs a plurality of tasks in one cycle according to the work procedure. Means.

As a result, the worker can confirm whether or not the standard time is being observed by checking the elapsed time display bar p6, which is useful for maintaining the work pace. Therefore, according to the work instruction device 10, it is possible to support the worker in observing the work time. The work instruction device 10 may alert the operator when the elapsed time during the work exceeds a predetermined ratio of the standard time of the work. The work instruction device 10 may call attention by changing the color of the work unit progress bar and the progress time, for example, or may call attention by causing the sound output device to output a sound. For example, when the elapsed time during work exceeds 70% of the standard time of the work, the work instruction device 10 sets the color of the work unit progress bar and the progress time to orange, and the elapsed time during the work is the standard time of the work. After that, the color of the work unit progress bar and the progress time may be changed to red to further call attention.

As shown in the explanatory image p3, when there are a plurality of types of washers and each of the plurality of types of washers is housed in a different case, the case containing the washer taken by the operator in the work is emphasized in the explanatory image p3. May be displayed.

The work instruction device 10 displays the work instruction image P corresponding to the next work on the display device 70 each time the current work is normally performed.

The work instruction device 10 may display the work result on the display device 70 when one cycle of work (a plurality of works assigned to the worker) is completed. FIG. 4D is a fourth diagram for explaining the basic operation of the work instruction device 10 according to the embodiment.

As shown in FIG. 4D, the work instruction device 10 may display the work time at the completion of one cycle on the display device 70. Further, the work instruction device 10 may further display the standard time of one cycle on the display device 70.

With reference to FIG. 4B again, the work instruction image P displays the actual information p7 indicating the actual work. FIG. 4B shows an example in which the work is a work using a driver, and the work instruction image P displays actual information p7 indicating the actual results in the work using the driver. That is, the work instruction image P including the actual result information p7 shown in FIG. 4B is an image at the time of driver cooperation, which is displayed when the work using the driver is performed. As the actual information p7, for example, the work result (“OK” shown in FIG. 4B), the driver name used for the work (“driver X” shown in FIG. 4B), the pulse, the count, and the number of NGs are displayed.

The work result shows the judgment result of whether or not the screw tightening is successful. In the work result, for example, every time the control unit 12 determines that the screw is properly tightened or not tightened, the determination result for the screw tightening is displayed. The work result is displayed for each screw, for example.

The driver name indicates information that identifies the driver used in the work using the driver. Only the driver displayed in the driver name is ready for operation.

The pulse is the actual number of pulses displayed for each screw tightening operation.

The count indicates the number determined by the control unit 12 that the screw has been tightened (for example, determined to be OK). Specifically, the count includes the actual number of screws determined to be "OK" ("0" shown in FIG. 4B) and the planned number of screws to be attached in the work ("0" shown in FIG. 4B). 3 ")" is displayed.

The number of times of NG is the number of times that the tightening of the screw fails. The number of NGs is counted up when the control unit 12 determines that the screw tightening has failed.

By displaying the actual result information p7 in this way, the operator can confirm the actual result in the work using the driver. Further, since the work result, the driver name, and the like are displayed on the display device 70, the worker can easily know the work result and the information of the driver to be used.

The actual information p7 in the work without the driver includes information indicating that the work currently being performed was normally performed, information indicating that the work was not performed normally, and the work being performed. Includes information indicating that it did not exist.

Further, the work instruction image P displays an operation image p8 for performing an operation such as switching the explanatory image p3. For example, when a predetermined button of the operation image p8 is selected with a mouse or the like, the explanatory image p3 or the like is switched according to the button. The buttons of the operation image p8 are, for example, a button for stopping the currently displayed explanatory image p3 or restarting the stopped explanatory image p3, a button for advancing to the next work, a button for returning to the previous work, and a reading of voice instructions. Includes start or stop buttons and the like.

In addition, a shortcut key may be assigned instead of the above button so that the keyboard can be easily operated. Further, the work instruction system 1 is provided with a sound collecting device (not shown, for example, a microphone), and the work instruction device 10 is displayed on the work instruction image P based on the voice of the worker acquired by the sound collecting device. You may change the contents.

As a result, the worker can change the display content of the work instruction image P or the like according to the progress of the work or the like. For example, by displaying the explanation image p3 of the difficult-to-understand part again, it is possible to prevent work mistakes. Further, for example, when the work is completed before the explanation of the explanatory image p3 is completed, the work can be smoothly proceeded by proceeding to the next work.

Further, the work instruction image P may display information such as a work list p9 showing a list of elemental works in one cycle.

This allows the worker to know information about the work performed before the current work and the work performed after the current work.

When the work instruction device 10 detects that the work currently being performed has been completed (for example, it detects based on the analysis result from the image pickup device 40), the work instruction device 10 next performs the information to be output to the display device 70. Automatically switches to the information of. For example, the work instruction device 10 automatically switches the work instruction image P displayed on the display device 70 to the work instruction image P of the work to be performed next. The work instruction device 10 is not limited to automatically switching the work instruction image P and the like, and the work instruction image P and the like may be switched by an operation from the operator (for example, operation of the foot switch).

[3. Work instruction system operation]
Next, the operation of the work instruction system 1 as described above will be described with reference to FIG. FIG. 5 is a sequence diagram showing the operation of the work instruction system 1 according to the embodiment. In the following, an example in which the prohibited area is not set will be described. Further, it is assumed that the work procedure includes work 1 to work n. The Nth work is also described as work N. Further, in FIG. 5, an example in which the image pickup device 40 (for example, the camera control unit 50) performs image analysis based on the image data will be described, but the image pickup device 40 is not limited to the image analysis.

As shown in FIG. 5, the work instruction device 10 reads the work procedure (1 to n) by activating the dedicated application and inputting predetermined information by the operation of the worker (S11). By inputting the "worker" and the "product number", the work instruction device 10 can determine whether or not the worker is a worker who performs the work of the serial number. The work instruction device 10 takes out a work instruction image P, a work table T1, and a detection table T2 according to predetermined information from a plurality of work instruction images and information tables stored in the storage unit 13. The reading includes reading the work instruction image P, the work table T1, and the detection table T2 from an external recording medium (CD-ROM, USB memory, etc.) of the work instruction device 10. Further, reading includes that the communication unit 11 acquires the work instruction image P, the work table T1 and the detection table T2 by communication from an external device, and also includes reading the work procedure.

Next, the work instruction device 10 reads the information of the work N and outputs it to the display device 70 (S12). Work N is work to be performed from now on. In step S12, the control unit 12 acquires the work instruction image P corresponding to the work N and the information (for example, the detection camera) about the camera operated in the work N from the detection table T2. Then, the control unit 12 outputs the work instruction image P corresponding to the work N to the display device 70.

The display device 70 acquires the information of the work N from the work instruction device 10 and displays the information (S13). As a result, the operator can perform the work after confirming the display device 70 or while confirming the display device 70. For example, when the work instruction system 1 includes a sound output device as an output device, a voice based on the acquired information may be output in step S13.

Next, the work instruction device 10 outputs the control information in the work N to the camera control unit 50 (S14). Specifically, the work instruction device 10 generates control information including information for specifying the camera to be operated in the work N (that is, the camera that captures the detection area) based on the detection table T2, and the camera. Output to the control unit 50. The image obtained by operating the camera may be a still image or a moving image.

In the present embodiment, when the work 1 is described as an example, the work instruction device 10 operates the camera a to take a picture of the detection area Ra, and stops taking pictures of the detection areas Rb to Rd in other work. The control information indicating that the operations of the cameras b to d are stopped is generated and output to the camera control unit 50. This enables the detection of the first component (eg, bis A) in the first region (eg, detection region Ra) and the second component (eg, bis B) in the second region (eg, detection region Rb). ) Is an example of disabling the detection.

The camera control unit 50 acquires control information and controls the image pickup unit 60 based on the acquired control information (S15). In the present embodiment, the camera control unit 50 controls the image pickup unit 60 so as to operate only the camera corresponding to the work N based on the control information. The control of the imaging unit 60 by the camera control unit 50 is not limited to this. The camera control unit 50 may operate a camera other than the camera corresponding to the work N.

The imaging unit 60 operates only the camera corresponding to the work N (S16), and outputs image data (an example of the sensing result) showing an image of the detection area corresponding to the work N to the camera control unit 50 (S17). .. Further, the sensing result includes, for example, whether or not a part of the detection region is shielded from light and at least one of the number of times the light is shielded when the sensor corresponding to the work N is an optical sensor. The output of the sensing result may be sequentially executed or may be executed at predetermined time intervals.

The camera control unit 50 acquires image data from the image pickup unit 60 (S18) and analyzes the acquired image data (S19). The camera control unit 50 detects the presence or absence of parts in the work by image analysis of the acquired image data. Then, the camera control unit 50 outputs the image data and the analysis result to the work instruction device 10 (S20). The output of the image data and the analysis result may be executed sequentially or may be executed at predetermined time intervals.

When the camera control unit 50 is also operating a camera other than the camera corresponding to the work N in step S15 (for example, when all the cameras a to d are being operated), the camera control unit 50 is operating the cameras a to d in step S20, respectively. Image analysis may be performed only on the image data of the camera corresponding to the work N from the image data of the above. In this way, the camera control unit 50 may extract the image data of the camera corresponding to the work N from the image data of the plurality of cameras acquired from the image pickup unit 60 and perform image analysis of the extracted image data. Good. Then, the camera control unit 50 may output only the image data and the analysis result corresponding to the work N to the work instruction device 10. Outputting only the image data and the analysis result of the camera corresponding to the work N to the work instruction device 10 by the camera control unit 50 is the first component (for example, the detection area corresponding to the work N) in the first region (for example, the detection region corresponding to the work N). Detection of the second part (for example, the part corresponding to the work other than the work N) in the second area (for example, the area corresponding to the work other than the work N) while enabling the detection of the part corresponding to the work N) Is an example of invalidating.

The camera control unit 50 does not have to analyze the image data. The camera control unit 50 may function as, for example, a relay device that acquires image data from the image pickup unit 60 and outputs the acquired image data to the work instruction device 10. When the camera control unit 50 is also operating a camera other than the camera corresponding to the work N in step S15 (for example, when all the cameras a to d are being operated), in step S20, all the cameras a to From the image data of d, only the image data of the camera corresponding to the work N may be output to the work instruction device 10.

In this way, the camera control unit 50 may extract the image data of the camera corresponding to the work N from the image data of the plurality of cameras acquired from the image pickup unit 60 and output the image data to the work instruction device 10. Outputting only the image data of the camera corresponding to the work N to the work instruction device 10 by the camera control unit 50 enables the detection of the first component in the first region and the detection of the second component in the second region. Is an example of invalidating.

Further, when the camera control unit 50 has a camera that captures an image having an angle of view shown in FIG. 3, the camera control unit 50 has a detection area corresponding to work N in the image data (for example, a detection area in the case of work 1). By performing image analysis only in Ra), it may be detected whether or not a component corresponding to work N is mounted in the detection area, and the detection result may be output to the work instruction device 10. In other words, in the work N, the detection result of whether or not the component corresponding to the work N is mounted in the detection area different from the detection area corresponding to the work N is not output to the work instruction device 10. When the camera control unit 50 outputs only the detection result of the detection area corresponding to the work N from the image data to the work instruction device 10, the detection of the first component in the first area is valid, and the second in the second area. This is an example of disabling the detection of two parts. Further, when the camera control unit 50 has a camera that captures an image having an angle of view shown in FIG. 3, the camera control unit 50 cuts out a detection area corresponding to work N from the image data, and uses only the cut out image data as a work instruction device. It may be output to 10.

Next, the work instruction device 10 acquires the image data and the analysis result corresponding to the work N from the camera control unit 50 (S21). The work instruction device 10 determines whether or not the work N is normally performed based on the acquired image data and the analysis result (S22). The determination will be described later. Then, the work instruction device 10 outputs the determined determination result to the display device 70 (S23). The determination result (for example, information indicating that the work N has been performed normally) is included in the work content. The work content according to the present embodiment includes at least a determination result.

The display device 70 acquires the determination result and displays the acquired determination result (S24). As a result, the work instruction device 10 can inform the worker whether or not the work N has been normally performed.

The processes of steps S12 to S24 are repeatedly executed until all the work procedures (for example, work 1 to work n) included in the work table T1 are completed. When it is determined in step S22 that the work N is not normally performed, the work instruction device 10 outputs a determination result indicating that the work N is not normally performed to the display device 70. Then, the work instruction device 10 does not execute the processing after step S12 in the next work. That is, no further work is performed.

In the above description, the work instruction device 10 has described an example in which the determination result is directly output to the display device 70. However, the work instruction device 10 may be used as a relay device (for example, a server device) for relaying communication between the work instruction device 10 and the display device 70. It may be output. Outputting the determination result to the relay device by the work instruction device 10 is included in outputting the determination result to the display device 70.

The work instruction device 10 may display the determination result on the display device 70 by superimposing it on the work instruction image P in the work.

Next, the operation of the work instruction device 10 will be described with reference to FIG. FIG. 6 is a flowchart showing the operation of the work instruction device 10 according to the embodiment. Each step shown in FIG. 6 is an example of a display step for displaying the work contents for the operator.

As shown in FIG. 6, the control unit 12 reads the work procedure (S31). Step S31 corresponds to step S11 shown in FIG. Then, the control unit 12 sets the work N to be performed to the work 1 in the read work procedure. That is, the control unit 12 sets N = 1 (S32). In steps S33 and subsequent steps, a case where N = 1 will be mainly described.

Next, the control unit 12 reads the information of the work N and outputs the read information of the work N to the display device 70 (S33). When the work N is the work 1, the control unit 12 outputs the work instruction image P corresponding to the work 1 to the display device 70.

Next, the control unit 12 effectively sets the detection area (an example of the first area) of the work N (S34). Enabling the detection area of the work N means that the work instruction device 10 makes a determination in step S36 using the information based on the image data of the camera that captures the detection area. In other words, it means that the determination in step S36, which will be described later, is performed without using the image data of the camera that captures the detection region (an example of the second region) other than the detection region of the work N. Step S34 is an example of enabling the detection of the first component in the first region and invalidating the detection of the second component in the second region.

Specifically, the control unit 12 generates control information based on the detection table T2 and outputs it to the image pickup apparatus 40. When the work N is the work 1, the control information here includes, for example, information for identifying the camera a, which is a camera for acquiring image data in the work 1. The control information may include, for example, an instruction to operate only the camera a among the cameras a to d. Step S34 corresponds to step S14 shown in FIG.

Next, the control unit 12 acquires the detection result for the work N (S35). The control unit 12 acquires image data and analysis results as detection results, for example. When the work N is the work 1, the control unit 12 acquires, for example, image data showing an image taken by the camera a and an analysis result in the image data. The control unit 12 may acquire at least one of the image data and the analysis result from the image pickup apparatus 40. Hereinafter, an example in which the control unit 12 acquires only the image data from the image pickup device 40 and acquires the analysis result in the own device based on the acquired image data will be described, but only the analysis result is acquired from the image pickup device 40. You may. Step S35 corresponds to step S21 shown in FIG. Note that step S35 may be performed a plurality of times for each operation, for example.

Next, the control unit 12 determines whether or not the component is mounted in the detection region based on the image data (S36). When the work N is the work 1, the control unit 12 determines whether or not the screw A is attached in the detection area Ra. The control unit 12 determines, for example, whether or not the screw A is attached to the detection area Ra by performing image analysis on the image data of the camera a. The method of image analysis is not particularly limited, and conventional techniques may be used. Step S36 corresponds to step S22 shown in FIG.

For example, when the image data indicates that the screw A is detected in the detection area Ra corresponding to the work 1, the control unit 12 determines that the screw A is attached to the detection area Ra. When it is determined that the screw A is attached to the detection area Ra (Yes in S36), the control unit 12 outputs information indicating that the work 1 has been normally performed (S37). The control unit 12 outputs the information to the display device 70, for example. Step S37 corresponds to step S23 shown in FIG. Further, the information indicating that the work 1 has been performed normally is an example of the contents of the first work.

Then, the control unit 12 determines whether or not all the work included in the read work procedure has been completed. Specifically, the control unit 12 determines whether or not N = n (S38). When N = n (Yes in S38), the control unit 12 ends the process. If the control unit 12 does not have N = n (No in S38), the control unit 12 proceeds to the next operation (S39). Specifically, the control unit 12 sets N = n + 1 and continues the processing after step S33. For example, work 2 is performed after work 1. For example, the information indicating that the work 2 in step S37 has been performed normally is an example of the content of the second work.

As a result, when a part corresponding to the work N (an example of the first part) is detected in the detection area (an example of the first area) in the work N (an example of the first work), the next work (an example of the first part) is performed. Information for performing 2 operations) is output to the display device 70. In other words, when the component corresponding to the work N is not detected in the detection area in the work N, the information for performing the next work is not output to the display device 70.

Further, when it is determined that the control unit 12 has not passed through the detection region (No in S36), the control unit 12 further determines whether or not a predetermined time has elapsed (S40). The predetermined period is the elapsed time from the start of the work N, and may be, for example, the elapsed time from the time when the detection area of the work N is effectively set in step S34.

When the predetermined time has elapsed (Yes in S40), the control unit 12 outputs information indicating that the work N has not been performed (S41). The control unit 12 outputs the information to the display device 70, for example. Then, the control unit 12 ends the process. Step S41 corresponds to step S23 shown in FIG. If the predetermined time has not elapsed (No in S40), the control unit 12 returns to step S35 and continues the process. The information indicating that the work N was not performed is an example of the first work content.

FIG. 7A is a diagram showing how the detection of the first component (for example, screw A) in the first region (for example, detection region Ra) is effective. Specifically, FIG. 7A shows how the camera a is photographing the detection area Ra in order to determine whether or not the screw A is attached to the detection area Ra in the operation 1. The diagonal hatching in FIG. 7A indicates a detection area in which detection is effective.

As shown in FIG. 7A, the main body 100 is equipped with only the screw A, which is a component corresponding to the work 1 among the work 1 to the work 4. At this time, the camera control unit 50 controls the image pickup unit 60 so that the detection of the screw A in the detection region Ra is effective. The camera control unit 50 operates only the camera a among the cameras a to d, for example. Thereby, it can be determined whether or not the work 1 is normally performed based on whether or not the screw A is attached to the detection area Ra.

For example, in work 1, when each of the cameras a to d is operating and it is determined in each of the detection areas Ra to Rd whether or not a component is mounted, in the area other than the detection area Ra. Is determined that the parts are not mounted, and an erroneous determination that the work 1 is not performed normally may be performed. According to the work instruction device 10 according to the present embodiment, since the detection is not performed in the area other than the detection area Ra, it is possible to suppress such an erroneous determination.

Next, a case where the work 4 is performed after the operations 1 to 3 are performed will be described. FIG. 7B is a diagram showing how the detection of the screw D in the second region (for example, the detection region Rd) is effective.

As shown in FIG. 7B, screw A, screw B, plate C, and screw D, which are parts corresponding to work 1 to work 4, are mounted on the main body 100. At this time, the camera control unit 50 controls the image pickup unit 60 so that the detection of the screw D in the detection region Rd is effective. The camera control unit 50 operates only the camera d among the cameras a to d, for example. Thereby, it can be determined whether or not the work 4 is normally performed based on whether or not the screw D is attached to the detection area Rd.

For example, in work 4, when each of the cameras a to d is operating and it is determined in each of the detection areas Ra to Rd whether or not a component is mounted, the camera is covered with the plate C. Since the existing screw A cannot be detected by the camera a, it may be determined that the screw A is not attached, and an erroneous determination that the work 1 is not performed normally may be performed. Therefore, as described above, in the work 4, it is preferable that only the screw D in the detection region Rd corresponding to the work 4 is detected.

In step 4, it may be determined whether or not each of the cameras a to d is operating and whether or not the component is mounted in each of the detection areas Ra to Rd. In this case, only the determination result of the screw D corresponding to the operation 4 is adopted, and it is determined whether or not the operation 4 is normally performed. This also enables the detection of the screw D (an example of the first component) in the detection area Rd (an example of the first region), and also enables the detection of the screw A (an example of the second component) in the detection area Ra (an example of the second region). ) Is an example of disabling the detection.

As described above, the work instruction device 10 effectively sets the detection of the parts corresponding to the work N in the detection area corresponding to the work N, and the work N is normally performed based on the image data obtained by photographing the detection area. Determine if it has been done. Further, the work instruction device 10 invalidates the detection of the parts corresponding to the work in the detection area other than the detection area corresponding to the work N. That is, the work instruction device 10 makes the above determination without using the image data of the detection area other than the detection area corresponding to the work N.

[4. Effect etc.]
As described above, in the work instruction method (an example of the work display method) according to the present embodiment, the first component is mounted in the first region and the first component is in the second region different from the first region. A work of performing a plurality of works including a first work in which the second component is not mounted and a second work in which the second component is mounted in the second area and the second component is not mounted in the first area. This is a work display method for displaying the work contents to a person. The work display method includes a first display step for displaying the first work content in the first work and a second display step for displaying the second work content in the second work. In the first display step, the detection of the first component in the first region is enabled and the detection of the second component in the second region is invalidated (S34), which is acquired from the image pickup unit 60 (an example of the sensor unit). When the first component is detected in the first region based on the image data (an example of the sensing result) (Yes in S36), the information indicating that the first work is normally performed is displayed as the first work content. Output to the device 70 (S37). In the second display step, the detection of the second component in the second region is enabled and the detection of the first component in the first region is disabled (S34), and the information is acquired from the imaging unit 60 (an example of the sensor unit). When the second component is detected in the second region based on the sensing result (Yes in S36), information indicating that the second work has been performed normally is output to the display device 70 as the content of the second work (Yes). S37).

For example, when the first operation is performed, if the detection in the second region is effective, it is determined whether or not the first operation is normally performed based on the image data of each of the first region and the second region. Will be done. In this case, even if the first work is normally performed, the part corresponding to the second work is not detected in the second area, or the part corresponding to the first work is detected in the second area. In that case, it may be determined that the first work has not been performed normally. That is, it may be erroneously detected.

On the other hand, in the work instruction method (an example of the work display method) according to the present embodiment, since the detection in the first region is invalid in the second work, it is determined whether or not the first work is normally performed. On the other hand, false detection can be suppressed. For example, when the second work is performed after the first work, at least a part of the first component of the first work may be hidden in the work performed between the first work and the second work. If the parts of the first work are detected in the second work in that state, the parts may not be detected accurately and may be erroneously detected. However, according to the work instruction method according to the present embodiment. For example, it is possible to suppress the occurrence of such false positives.

Further, in the work instruction method according to the present embodiment, since the detection in the second region is invalid in the first work, the process of determining whether or not the first work in the first work is normally performed is performed. The amount of processing can be reduced. For example, it is possible to reduce the load of image processing for determining whether or not the first operation is normally performed based on the image captured by the camera.

Therefore, according to the work instruction method according to the present embodiment, it is possible to reduce the processing load in the determination process and suppress erroneous detection.

Further, the sensor unit has an imaging unit 60 that captures an image including the first region and the second region as a sensing result.

Thereby, it is possible to determine whether or not the work is normally performed based on the image data (an example of the sensing result) showing the image taken by the imaging unit 60. For example, when the image analysis result of the image data (for example, the position, shape, color, etc. of the part) is used, it is possible to more accurately determine whether or not the work has been performed normally. In other words, according to the work display method according to the present embodiment, erroneous detection can be further suppressed.

Further, the second work is a work performed after the first work, and in the first display step, an image for performing the first work is further displayed on the display device 70, and in the second display step, the image for performing the first work is displayed. Further, in the first display step, when the first component is detected in the first region, an image for performing the second operation is displayed on the display device 70.

As a result, when the first work is normally performed, an image for performing the next work is automatically displayed, so that the work efficiency is improved.

Further, as described above, in the work instruction device 10 (an example of the work display device) according to the present embodiment, the first component is mounted in the first area and the second area different from the first area is used. A plurality of operations including a first operation in which the first component is not attached and a second operation in which the second component is attached to the second area and the second component is not attached to the first area. It is a work display device for displaying the work contents to the worker who performs the work. When the first work is performed, the work instruction device 10 enables the detection of the first component in the first region, invalidates the detection of the second component in the second region, and performs the second work. In this case, the control unit 12 that enables the detection of the second component in the second region and invalidates the detection of the first component in the first region, and the image pickup unit 60 (when the first operation is performed) When the first component is detected in the first region based on the image data (an example of the sensing result) acquired from the sensor unit), the information indicating that the first operation is normally performed is provided in the first information. The second work is performed when the second component is detected in the second region based on the image data acquired from the image pickup unit 60 when the work content is output to the display device 70 and the second work is performed. It is provided with a communication unit 11 (an example of an output unit) that outputs information indicating that the above is normally performed to the display device 70 as the second work content.

As a result, the same effect as the above-mentioned work instruction method is obtained.

(Modification 1 of the embodiment)
Next, the work instruction method according to the first modification of the embodiment will be described with reference to FIGS. 8 and 9. FIG. 8 is a flowchart showing the operation of the work instruction device 10 according to the first modification of the embodiment. Each step shown in FIG. 8 is an example of a display step for displaying the work contents for the operator.

The configuration of the work instruction device according to this modification is the same as that of the work instruction device 10 according to the embodiment, and the description thereof will be omitted. Further, also in the flowchart of FIG. 8, the same reference numerals are given to the same processes as those of the flowchart of FIG. 6, and the description thereof will be omitted or simplified.

As shown in FIG. 8, the control unit 12 effectively sets the detection area and the prohibited area of the work N after reading the work N in step S33 (S134). Enabling the detection area and the prohibited area of the work N means that the determinations of steps S136 and S142, which will be described later, are performed using the image data of the camera that captures the detection area and the prohibited area of the work N. Step S134 enables the detection of the first component (for example, screw A) in the first region (for example, detection region Ra) and enables the detection of the second component (for example, plate) in the second region (for example, detection region Rc). This is an example of invalidating the detection of C) and further enabling the detection of the first component in the third region (for example, detection regions Rb and Rd) which is a prohibited region.

Specifically, the control unit 12 generates control information based on the detection table T2 and outputs it to the image pickup apparatus 40. When the work N is the work 1, the control information here may include an instruction to operate only the cameras a, b, and d among the cameras a to d. The control information may include an instruction to output the image data of the camera a, which is the detection camera corresponding to the work 1, and the cameras b and d, which are the prohibited cameras corresponding to the work 1, to the work instruction device 10. Step S134 corresponds to step S14 shown in FIG.

Next, the control unit 12 acquires the detection result for the work N from the image pickup apparatus 40 (S135). The control unit 12 acquires, for example, image data and analysis results. When the work N is the work 1, the control unit 12 acquires the image data of each of the cameras a, b, and d and the analysis result in the image data. The control unit 12 may acquire at least the image data out of the image data and the analysis result from the image pickup apparatus 40 as the detection result. Hereinafter, an example in which the control unit 12 acquires only image data from the image pickup apparatus 40 in step S135 will be described. Step S135 corresponds to step S21 shown in FIG. Note that step S135 may be performed a plurality of times for each operation, for example.

Next, the control unit 12 determines whether or not the component is mounted in the detection region based on the image data (S36). When the control unit 12 determines that the component is mounted in the detection area (Yes in S36), the control unit 12 further determines whether or not the component is mounted in the prohibited area based on the image data (S142). Specifically, the control unit 12 determines whether or not the component corresponding to the work N is mounted in the prohibited area based on the image data of the prohibited camera. When the work N is the work 1, the control unit 12 prohibits the image data of the cameras b and d when it indicates that the screw A is not detected in the detection areas Rb and Rd which are the prohibited areas corresponding to the work 1. It is determined that the screw A is not attached to the area. The image data used for the determination in steps S36 and S142 may be, for example, image data indicating images taken at the same time. Steps S36 and S142 correspond to step S22 shown in FIG.

Here, the detection area and the prohibited area in the work 1 will be described with reference to FIG. FIG. 9 is a diagram showing how the prohibited area corresponding to the work 1 is effective. The downward-sloping diagonal line shown in FIG. 9 indicates the detection area in the work 1, and the downward-sloping diagonal line indicates the prohibited area in the work 1.

As shown in FIG. 9, in the operation 1, the detection of the screw A in the detection area Ra is effectively set. Thereby, in the work 1, it is possible to determine whether or not the screw A is attached to the detection area Ra (determination shown in S36) based on the image data of the camera a. In the example of FIG. 9, since the screw A is mounted on the detection area Ra of the main body 100, the work instruction device 10 determines Yes in step S36.

Further, in the work 1, the detection of the screw A in the detection areas Rb and Rd is effectively set. As a result, in the work 1, based on the image data of the cameras b and d, it is determined whether or not the screw A is erroneously attached to the detection areas Rb and Rd which are the prohibited areas in the work 1 (determination shown in S142). It can be carried out. In the example of FIG. 9, since the screw A is not attached to the prohibited areas (detection areas Rb and Rd) in the work 1 of the main body 100, the work instruction device 10 determines No in step S142.

With reference to FIG. 8 again, when it is determined that no component is mounted in the prohibited area (No in S142), the control unit 12 outputs information indicating that the work N has been performed normally (S37). .. The control unit 12 outputs the information to the display device 70, for example.

Further, when it is determined that the component is mounted in the prohibited area (Yes in S142), the control unit 12 outputs information indicating that the work N has not been normally performed (S143). Then, the control unit 12 ends the process. In addition, steps S37, S41, and S143 correspond to step S23 shown in FIG. Further, for example, the information indicating that the work 1 was not normally performed is an example of the first work content.

As described above, the work instruction device 10 effectively sets the detection area and the prohibited area according to the work, and is based on the image data of only the camera that shoots the detection area and the camera that shoots the prohibited area. , Judge whether the work was done normally. Further, the work instruction device 10 disables detection of an area other than the detection area and the prohibited area according to the work.

As shown in FIG. 2B, the process of step S142 may not be performed in the work in which the prohibited camera is not set (for example, works 3 and 4). In this case, if Yes in step S36, the process proceeds to step S37. For example, if Yes in step S36, the control unit 12 executes the determination process of step S142 based on the detection table T2 (for example, based on whether or not a prohibited camera is set in the work). It may be determined whether or not to do so.

As described above, the work instruction method (an example of the work display method) according to this modification is a third area different from the first area and the second area, and mounting of the first component is prohibited in the first work. In the first display step, the detection of the first component in the third region is further enabled (S134), and the image data (sensing) acquired from the imaging unit 60 (an example of the sensor unit) includes the third region. When the first component is not detected in the third region based on an example of the result), information indicating that the first work has been normally performed is output to the display device 70 as the first work content (S37). ..

As a result, it is possible to determine whether or not the work has been performed normally, based on whether or not the first component has been detected in the prohibited area. For example, when the first component is detected only in the first region of the first region and the third region, the operator is performing the correct work. Therefore, according to the work display method according to the present modification, if the first component is detected in the first region and is not detected in the third region, it is determined that the work has been performed normally. Therefore, it is possible to more accurately determine whether or not the work has been performed normally.

Further, in the first display step, when the first component is detected in the third region (Yes in S142), information indicating that the first work was not normally performed is displayed as the first work content. Output to the device 70 (S143).

This makes it possible to notify the worker that the work has not been performed normally. For example, if the first component is detected in both the first region and the third region, the operator may be performing an erroneous operation (for example, the first component is mounted in the third region). There is sex. Therefore, according to the work display method according to the present modification, when the first component is detected in the third region, that is, when there is a possibility that an erroneous work is being performed, the first component is in the first region. Even if it is detected in, it is determined that the work was not performed normally. Therefore, it is possible to more accurately determine whether or not the work has been performed normally.

In addition, when the information indicating that the work has not been performed normally includes the information indicating that the first component has been detected in the third region (prohibited region), the operator can see that the first component is in the third region. It is possible to know that it was determined that the work was not performed normally because it was detected in. For example, when the third area is an area for mounting a component similar to the first component, the operator can confirm whether or not the component has been mistakenly picked up.

(Modification 2 of the embodiment)
Next, the work instruction method according to the second modification of the embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing the operation of the work instruction device 10 according to the second modification of the embodiment. Each step shown in FIG. 10 is an example of a display step for displaying the work contents for the operator.

The configuration of the work instruction device according to this modification is the same as that of the work instruction device 10 according to the embodiment, and although description thereof will be omitted, each of the cameras a to d is, for example, a color camera. Further, also in the flowchart of FIG. 10, the same reference numerals are given to the same processes as those of the flowchart of FIG. 6, and the description thereof will be omitted or simplified.

As shown in FIG. 10, the control unit 12 effectively sets the detection region of the work N in step S34, and then acquires the detection result for the work N from the image pickup apparatus 40 (S335). The control unit 12 acquires, for example, image data and analysis results. When the work N is the work 1, the control unit 12 acquires the image data of the camera a and the analysis result in the image data. The analysis result here includes, for example, information on the color of the component mounted in the detection area. That is, in step S335, the control unit 12 acquires information on the color of the component as an analysis result. The analysis result may be acquired from the image pickup apparatus 40, or may be acquired by performing image processing on the image data acquired from the image pickup apparatus 40 by the own apparatus. Step S335 corresponds to step S21 shown in FIG.

Next, the control unit 12 determines whether or not the component is mounted in the detection region based on the image data (S36). When it is determined that the component is mounted in the detection area (Yes in S36), the control unit 12 further determines whether or not a predetermined color has been detected based on the analysis result (S242). The control unit 12 determines whether or not a predetermined color has been detected based on, for example, the analysis result and the detection table T2.

Explaining the work 1 as an example, the control unit 12 determines when the color of the screw A mounted on the detection area Ra corresponding to the work 1 is the same as the detection color “blue” shown in the detection table T2. It is determined that the color has been detected. Steps S36 and S242 correspond to step S22 shown in FIG.

When the control unit 12 detects a predetermined color (Yes in S242), the control unit 12 proceeds to step S37 and executes the subsequent processing. Further, when the control unit 12 does not detect a predetermined color (No in S242), the control unit 12 outputs information indicating that the work N has not been normally performed (S243). Then, the control unit 12 ends the process. In addition, steps S37, S41, and S243 correspond to step S23 shown in FIG.

As described above, the work instruction device 10 effectively sets the detection of the detection area according to the work, and whether or not the component is mounted in the detection area according to the work and whether or not the predetermined color is detected. Therefore, control is performed to determine whether or not the work has been performed normally.

In the work in which the detection color is not set, if Yes is determined in step S36, the process proceeds to step S37. The control unit 12 determines whether or not to execute the determination process of step S242 based on, for example, the detection table T2 (for example, based on whether or not the detection color is set in the work). May be good.

As described above, the image in the work instruction method (an example of the work display method) according to the present modification includes information on colors in the first region and the second region.

As a result, it is possible to determine whether or not the work has been performed normally using the color information, so that the above determination can be performed more accurately. Further, for example, when a prohibited area is set and a predetermined color is not detected in the prohibited area, it is determined that the work has been performed normally, so that erroneous detection can be further suppressed.

In addition, the color information includes the color of the part used in at least one of the first operation and the second operation.

Thereby, for example, when a part having a shape similar to that used by the operator in the work but having a different color is mistakenly attached, it can be detected by the color of the part. Therefore, it is possible to more accurately determine whether or not the work has been performed normally.

(Other embodiments)
Although the embodiments and the modifications (hereinafter, also referred to as the embodiments and the like) have been described above, the present invention is not limited to such embodiments and the like.

For example, in the above-described embodiment and the like, an example in which one detection area is used for one work has been described, but there may be a plurality of detection areas.

Further, in the above-described embodiment and the like, an example in which a detection area other than the current work is set as the prohibited area has been described, but the present invention is not limited to this. The prohibited area may be an area different from the detection area. That is, the prohibited area and the detection area may be different areas from each other.

Further, the operation of the work instruction device 10 (operation shown in FIG. 6, FIG. 8 or FIG. 10) in the above-described embodiment or the like may be switched for each work based on the information included in the detection table T2. Good. For example, in tasks 1 and 2, since the prohibited camera is set, the determination process (processing after S36) shown in FIG. 8 is executed, and in tasks 3 and 4, the prohibited camera is not set. The determination process (process after S36) shown in FIG. 6 may be executed.

Further, in the above-described embodiment and the like, an example in which the camera control unit 50 performs image analysis processing on image data has been described, but the control unit 12 of the work instruction device 10 may perform the image analysis processing.

Further, in the above-described embodiment and the like, the work instruction device 10 has described an example in which the read information of the work N is output to the display device 70 (see, for example, S33 shown in FIG. 6), but the information is displayed on the display device. It is not necessary to perform the process of outputting to 70.

Further, in the above-described embodiment and the like, an example in which it is determined that the work has been normally performed when a component corresponding to the detection area is detected in the detection area has been described, but the present invention is not limited to this. When the component is a screw, in step S36, when the number of pulses in the screw tightening operation is within the upper and lower limit pulse numbers (see FIG. 2A) and the screw is detected in the detection region, the component is in the detection region. Is attached, that is, it may be determined that the work has been performed normally.

Further, in the above-described embodiment and the like, an example in which one component is attached to one work has been described, but the present invention is not limited to this. A plurality of parts (for example, a plurality of types of parts) may be attached to one operation. Further, although an example in which one component is associated with one detection area has been described, the present invention is not limited to this. A plurality of parts may be associated with one detection area. In the first operation, for example, a first component including two or more components may be attached to the first region. Then, the camera a may be provided so as to be able to detect the first component including two or more components.

Further, in the above-described embodiment and the like, an example in which the detection regions Ra to Rd are photographed by different cameras has been described, but the present invention is not limited to this. The work instruction system 1 has, for example, one camera rotatably supported in the horizontal direction and the vertical direction, and by rotating the camera according to the detection area, the detection areas Ra to Rd are separated. You may take a picture. As a result, the required number of cameras in the work instruction system 1 can be reduced.

Further, in the above-described embodiment and the like, mounting of the component has described an example in which the screw is screwed and the plate is mounted, but the present invention is not limited to this. The fact that the component is mounted means that, for example, the component is joined to the detection region of the main body 100 by a joining member (for example, solder), and the component is joined to the detection region of the main body 100 by a fastening member such as a screw. Including being fixed.

Further, in the above-described embodiment and the like, an example in which one display device 70 is connected to the work instruction device 10 has been described, but the present invention is not limited to this. For example, when an operator performs each of the operations 1 to 4 on different work tables or the like, the display device 70 may be arranged on each of the work tables.

Further, in the above-described embodiment and the like, the work performed by the worker has described an example including a work using a tool (for example, a driver) and a work not using a tool, but the work is not limited to this. The work performed by the worker may be, for example, only work that does not use a tool.

Further, in the above-described embodiment and the like, an example in which the tool is a driver has been described, but the present invention is not limited to this. The tool may be a tool that is communicably connected to the work instruction device 10 and can be controlled by the control information from the work instruction device 10. The tool may be, for example, a tool other than a screwdriver (for example, a power tool) or a jig (for example, an inspection jig).

Further, the division of the functional block in the block diagram shown in FIG. 1 is an example, and a plurality of functional blocks can be realized as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be divided into other functional blocks. It may be moved to a functional block. Further, the functions of a plurality of functional blocks having similar functions may be processed by a single hardware or software in parallel or in a time division manner.

Further, in the above-described embodiment and the like, each of the work instruction device 10 and the image pickup device 40 is realized by a single device, but may be realized by a plurality of devices connected to each other.

Further, the communication method between the devices included in the work instruction system 1 in the above-described embodiment and the like is not particularly limited. Wireless communication may be performed or wired communication may be performed between the devices. As the wireless communication system, for example, a wireless communication system such as WiFi (registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), etc. specified in IEEE802.5.1 is used.

Further, the order in which each step in the flowchart is executed is for exemplifying for concretely explaining the present invention, and may be an order other than the above. For example, a part of the above steps may be executed at the same time (parallel) as other steps, or may be executed in a different order from the other steps.

Further, some or all of the components included in the work instruction device 10 and the image pickup device 40 in the above-described embodiment may be composed of one system LSI (Large Scale Integration). .. For example, it may be composed of a system LSI having a control unit 12 of the work instruction device 10 and a processing unit such as a camera control unit 50 of the image pickup device 40.

A system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. It is a computer system composed of. A computer program is stored in the ROM. The system LSI achieves its function by operating the microprocessor according to the computer program.

Although it is referred to as a system LSI here, it may be referred to as an IC, an LSI, a super LSI, or an ultra LSI depending on the degree of integration. Further, the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of circuit cells inside the LSI may be used.

Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. There is a possibility of applying biotechnology.

Further, one aspect of the present invention may be a computer program that causes a computer to execute each characteristic step included in the work instruction method. Further, one aspect of the present invention may be a computer-readable non-temporary recording medium on which such a computer program is recorded.

In the above-described embodiment or the like, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

In addition, it is realized by subjecting various modifications to the embodiments that can be conceived by those skilled in the art, or by arbitrarily combining the components and functions of the embodiments without departing from the spirit of the present invention. Forms are also included in the present invention.

1 Work instruction system (work display system)
10 Work instruction device (work display device)
11 Communication unit (output unit)
12 Control unit 13 Storage unit 20 1st driver 30 2nd driver 40 Imaging device 50 Camera control unit 60 Imaging unit (sensor unit)
70 Display device 100 Main unit P Work instruction image p1 Work content p2 Precautions p3 Explanation image p4 Part name p5 Selected image p6 Elapsed time display bar p7 Actual information p8 Operation image p9 Work list Ra to Rd Detection area T1 Work table T2 Detection Table for

Claims (8)

The first work in which the first component is mounted in the first region and the first component is not mounted in the second region different from the first region, and the second component is mounted in the second region. In addition, it is a work display method for displaying work contents to a worker who performs a plurality of works including a second work in which the second component is not mounted in the first area.
The first display step for displaying the first work content in the first work and the second display step for displaying the second work content in the second work are included.
In the first display step,
The detection of the first component in the first region is enabled, and the detection of the second component in the second region is invalidated.
When the first component is detected in the first region based on the sensing result acquired from the sensor unit, information indicating that the first work has been normally performed is displayed as the first work content. Output to
In the second display step,
The detection of the second component in the second region is enabled, and the detection of the first component in the first region is invalidated.
When the second component is detected in the second region based on the sensing result acquired from the sensor unit, the information indicating that the second work has been normally performed is used as the content of the second work. Output to the display device,
Work display method. A third region different from the first region and the second region, including a third region in which mounting of the first component is prohibited in the first operation.
In the first display step, further
The detection of the first component in the third region is enabled, and the detection is enabled.
When the first component is not detected in the third region based on the sensing result acquired from the sensor unit, the information indicating that the first work has been normally performed is used as the first work content. Output to the display device,
The work display method according to claim 1. In the first display step, further, when the first component is detected in the third region, the display device uses information indicating that the first work has not been normally performed as the first work content. Output to
The work display method according to claim 2. The sensor unit includes an imaging device that captures an image including the first region and the second region as the sensing result.
The work display method according to any one of claims 1 to 3. The image contains information about colors in the first and second regions.
The work display method according to claim 4. The information regarding the color includes the color of the part used in at least one of the first operation and the second operation.
The work display method according to claim 5. The second work is a work performed after the first work.
In the first display step, an image for performing the first operation is further displayed on the display device.
In the second display step, when the first component is detected in the first region in the first display step, an image for performing the second operation is displayed on the display device.
The work display method according to any one of claims 1 to 6. The first work in which the first component is mounted in the first region and the first component is not mounted in the second region different from the first region, and the second component is mounted in the second region. In addition, it is a work display device for displaying work contents to a worker who performs a plurality of works including a second work in which the second component is not mounted in the first area.
When the first operation is performed, the detection of the first component in the first region is enabled, the detection of the second component in the second region is invalidated, and the second operation is performed. In this case, a control unit that enables detection of the second component in the second region and invalidates detection of the first component in the first region.
When the first work is performed and the first component is detected in the first region based on the sensing result acquired from the sensor unit, it indicates that the first work has been performed normally. When the information is output to the display device as the first work content and the second work is performed, the second component is detected in the second region based on the sensing result acquired from the sensor unit. Occasionally, it includes an output unit that outputs information indicating that the second work has been normally performed to the display device as the content of the second work.
Work display device.

Images