JPWO2020157821A1 - Work support system - Google Patents

Abstract

A camera (1) that captures the worker (11), a camera (2) that captures the work area, a projector (3) that projects an input image onto the work area, and a worker (1) based on the imaging results of the camera (1). A specific part (402) that specifies the position of 11), a specific part (403) that specifies the position of an object existing in the work area from the image capture result by the camera (2), and three-dimensional data that is an icon representing a work instruction. An extraction unit (404) that sequentially extracts images, and an image generation unit (405) that generates a pseudo three-dimensional image from three-dimensional data based on the specific result by the specific unit (402) and the specific result by the specific unit (403). , The output unit (407) that outputs data indicating a pseudo three-dimensional image to the projector (3), and the extraction unit (404) that confirms the completion of the work by the operator (11) and extracts the next three-dimensional data. It is provided with a confirmation instruction unit (408) for instructing.

Description

The present invention relates to a work support system for giving work instructions.

Conventionally, when performing work such as assembly work on a product, the worker performs the work after confirming the work instruction using a document such as a work procedure or a checklist or a monitor.

On the other hand, the document or monitor is placed in an area different from the work area where the product is placed. Therefore, the operator needs to take his eyes off the product and look back at the document or the monitor every time he confirms the work instruction. In addition, an area for placing documents or monitors is required in addition to the work area for placing products. In addition, characters are often used in work instructions by documents or monitors, but in this case, for example, in a factory where workers of different languages coexist, translation is required. Therefore, written work instructions are not desirable from the viewpoint of globalization.

On the other hand, a system for projecting a two-dimensional image such as a circle or an arrow on a specific part of a product is known (see, for example, Patent Document 1).
Further, there is also known a system in which an image is projected onto a device such as an HMD (Head Mounted Display) worn by an operator to give a work instruction.

International Publication 2016/084151

However, in the method of projecting a two-dimensional image such as a circle or an arrow on the product, the work location can be instructed, but it is difficult to instruct the specific work content such as how to assemble the parts. Therefore, in this system, work instructions by documents or monitors are indispensable separately.
Further, in the method of projecting an image on a device such as an HMD, it is necessary to prepare the device for each worker, and the work with the device attached is troublesome.

The present invention has been made to solve the above-mentioned problems, and it is possible to give a work instruction without characters while the operator is looking at the work area without wearing a special device. The purpose is to provide a support system.

The work support system according to the present invention includes a first imaging device that images a worker who works in the work area, a second imaging device that images the work area, a projection device that projects an input image onto the work area, and a first image. 1 A first specific part that identifies the position of an operator from the image pickup result by the image pickup device, a second specific part that specifies the position of an object existing in the work area from the image pickup result by the second image pickup device, and an icon representing a work instruction. The projected image is viewed from one direction from the extraction unit that extracts the three-dimensional data in order, and the three-dimensional data extracted by the extraction unit based on the specific result by the first specific unit and the specific result by the second specific unit. An image generation unit that generates a pseudo 3D image that can be viewed stereoscopically, an output unit that outputs data indicating the pseudo 3D image generated by the image generation unit to the projection device, and confirmation of work completion by the operator. It is characterized in that it is provided with a confirmation instruction unit that instructs the extraction unit to extract the next three-dimensional data.

According to the present invention, since it is configured as described above, it is possible to give a work instruction without characters while the operator is looking at the work area without wearing a special device.

It is a figure which shows the configuration example of the work support system which concerns on Embodiment 1. FIG. It is a figure which shows the configuration example of the arithmetic unit in Embodiment 1. FIG. It is a flowchart which shows the operation example of the arithmetic unit in Embodiment 1. FIG. 4A is a diagram showing an example of a work area before the work is performed, and FIG. 4B is a diagram for explaining the setting of the projection area by the image generation unit and the control unit in the first embodiment. FIG. 5A is a diagram showing an example of three-dimensional data representing a work instruction of the first work, and FIG. 5B is for explaining deformation of the three-dimensional data shown in FIG. 5A by the image generation unit in the first embodiment. It is a figure of. FIG. 6A is a diagram showing a state when an operator sees the projected image after a pseudo three-dimensional image representing a work instruction of the first work is projected by the projector according to the first embodiment, and FIG. 6B is a diagram. , Is a diagram showing a case where a third party other than the worker views the projected image from a direction different from that of the worker. It is a figure which shows the state when a worker sees a projected image after the pseudo three-dimensional image which shows the work instruction of a second work is projected by the projector in Embodiment 1. FIG. 8A and 8B are diagrams showing a hardware configuration example of the arithmetic unit according to the first embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1.
FIG. 1 is a diagram showing a configuration example of a work support system according to the first embodiment.
The work support system gives work instructions to the worker 11 who performs work such as assembly work on the product 12 in the work area. As shown in FIG. 1, the work support system includes a camera (first imaging device) 1, a camera (second imaging device) 2, a projector (projection device) 3, and an arithmetic unit 4. The product 12 may be placed on a trolley 13 and moved to a work area, or may be placed on a work surface 14 (see FIG. 4 or the like) provided in the work area. FIG. 1 shows a case where the product 12 is moved to the work area while being mounted on the trolley 13, and the worker 11 works on the product 12 mounted on the trolley 13. It is assumed that there is only one worker 11 who works in one work area.

The camera 1 takes an image of the worker 11 who works in the work area. At this time, it is preferable that the camera 1 captures at least a part of the body (for example, the eyes or the head) including the eyes of the worker 11. The camera 1 is arranged at a position where the region where the operator 11 is supposed to be located is included in the imaging region. When the area where the worker 11 is supposed to be located is wider than the imaging area, the camera 1 needs to have a mechanism for making at least one of the imaging position and the imaging angle variable. As the camera 1, for example, an infrared camera can be used. The image data obtained by the camera 1 is output to the arithmetic unit 4. Further, when the camera 1 has a mechanism for changing at least one of the imaging position and the imaging angle, the camera 1 adds at least one of the imaging position and the imaging angle at the time of imaging in addition to the above image data. Is output to the arithmetic unit 4. The arithmetic unit 4 holds in advance data indicating the initial imaging position and imaging angle of the camera 1.

The camera 2 captures the work area. As the camera 2, for example, an infrared camera can be used. The image data obtained by the camera 2 is output to the arithmetic unit 4. The arithmetic unit 4 holds in advance data indicating an imaging position and an imaging angle of the camera 2.

The projector 3 projects an input image onto the work area. When the working position of the worker 11 is fixed, the projection position and the projection angle of the projector 3 may be fixed, but when the working position of the worker 11 is not fixed, the projector 3 has the projection position and the projection angle. It has a mechanism that makes at least one of the projection angles variable. The arithmetic unit 4 holds in advance data indicating the initial projection position and projection angle of the projector 3.

The arithmetic unit 4 generates a pseudo three-dimensional image and controls the projector 3 based on the image pickup result by the camera 1 and the image pickup result by the camera 2. The pseudo three-dimensional image is an image representing a work instruction, and is an image that can be viewed stereoscopically when a person views the image (projected image) projected on the work area by the projector 3 from a specific direction. As shown in FIG. 2, the arithmetic unit 4 includes a recording unit 401, a specific unit (first specific unit) 402, a specific unit (second specific unit) 403, an extraction unit 404, an image generation unit 405, and a control unit 406. It has an output unit 407 and a confirmation instruction unit 408.

The recording unit 401 records three-dimensional data, which is an icon representing a work instruction. This three-dimensional data exists for each work process and is recorded in the recording unit 401 in association with the identification information indicating the work process order.
The recording unit 401 includes, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable ROM), or an EPROM (Electrically EPROM), or a magnetic disk. , Flexible discs, optical discs, compact discs, mini discs, DVDs (Digital Versailles Disc), and the like.

Note that FIG. 1 shows a case where the recording unit 401 is provided inside the work support system, but the present invention is not limited to this, and the recording unit 401 may be provided outside the work support system.

The identification unit 402 identifies the position of the worker 11 from the image pickup result by the camera 1. At this time, it is preferable that the specific unit 402 specifies the position of at least a part of the body (for example, eyes or head) including the eyes of the worker 11 as the position of the worker 11. When the camera 1 outputs data indicating at least one of the imaging position and the imaging angle in addition to the image data, the specifying unit 402 specifies the position of the operator 11 in consideration of the data. .. As a method of specifying the position of the worker 11 from the image, various conventionally known methods can be applied, and the description thereof will be omitted. Further, the specifying unit 402 may specify the direction of the worker 11 in addition to the position of the worker 11 from the image pickup result by the camera 1.

The identification unit 403 specifies the position and shape of an object existing in the work area from the image pickup result by the camera 2. Examples of the object include a product 12, a trolley 13, a work surface 14, and the like. As a method of specifying the position and shape of an object existing in the work area from an image, various conventionally known methods can be applied, and the description thereof will be omitted. The identification unit 403 may specify the orientation of the object in addition to the position and shape of the object existing in the work area from the image pickup result by the camera 2.

The extraction unit 404 extracts three-dimensional data from the recording unit 401 in order. Specifically, the extraction unit 404 first extracts the three-dimensional data associated with the identification information indicating the first work process from the recording unit 401. After that, each time the extraction unit 404 receives an instruction from the confirmation instruction unit 408, the extraction unit 404 extracts three-dimensional data associated with the identification information indicating the next work process in order.

The image generation unit 405 generates a pseudo three-dimensional image from the three-dimensional data extracted by the extraction unit 404 based on the specific result by the specific unit 402 and the specific result by the specific unit 403. At this time, the image generation unit 405 first sets the projection area by the projector 3 from the work area based on the specific result by the specific unit 402 and the specific result by the specific unit 403. It is assumed that the image generation unit 405 knows the positional relationship between the object existing in the work area and the work place. Then, the image generation unit 405 transforms the three-dimensional data extracted by the extraction unit 404. At this time, the image generation unit 405 is stereoscopically viewed when the projected image is viewed from the position of the worker 11 specified by the specific unit 402 when the pseudo three-dimensional image is projected from the projector 3 into the set projection area. As such, the three-dimensional data is transformed. Then, the image generation unit 405 generates a pseudo three-dimensional image by converting the deformed three-dimensional data into a two-dimensional image.

The control unit 406 controls at least one of the projection position and the projection angle of the projector 3 based on the specific result by the specific unit 402 and the specific result by the specific unit 403. At this time, the control unit 406 first sets the projection area by the projector 3 from the work area based on the specific result by the specific unit 402 and the specific result by the specific unit 403. The projection area set by the control unit 406 is the same as the projection area set by the image generation unit 405. It is assumed that the control unit 406 knows the positional relationship between the object existing in the work area and the work place. Then, the control unit 406 controls at least one of the projection position and the projection angle of the projector 3 so that the projection can be performed on the set projection area.

When the projection position and the projection angle of the projector 3 are fixed, the control unit 406 is unnecessary.

The output unit 407 outputs data indicating a pseudo three-dimensional image generated by the image generation unit 405 to the projector 3.

The confirmation instruction unit 408 confirms the completion of the work by the operator 11 and instructs the extraction unit 404 to extract the next three-dimensional data. Here, for example, the confirmation instruction unit 408 may confirm the work completion by the worker 11 by receiving the work completion notification input by the user via the input device (not shown). Further, the confirmation instruction unit 408 may confirm the completion of the work by the operator 11 by analyzing the image pickup result by the camera 2.
When the three-dimensional data representing the last work process is extracted by the extraction unit 404, the arithmetic unit 4 ends the process when the confirmation instruction unit 408 confirms the completion of the work by the worker 11. ..

Next, an operation example of the arithmetic unit 4 shown in FIGS. 1 and 2 will be described with reference to FIG. The recording unit 401 records three-dimensional data for each work process. Further, each of the camera 1 and the camera 2 constantly performs imaging, and outputs the obtained image data to the arithmetic unit 4.
In the operation example of the arithmetic unit 4, as shown in FIG. 3, first, the identification unit 402 specifies the position of the worker 11 from the image pickup result by the camera 1 (step ST301). Here, the identification unit 402 specifies the position of at least a part of the body including the eyes of the worker 11 as the position of the worker 11. The process by the specific unit 402 is performed every time the image data is input from the camera 1 to the arithmetic unit 4.

Further, the identification unit 403 specifies the position and shape of the object existing in the work area from the image pickup result by the camera 2 (step ST302).

Further, the extraction unit 404 extracts three-dimensional data from the recording unit 401 (step ST303). At this time, the extraction unit 404 extracts the three-dimensional data associated with the identification information indicating the first work process from the recording unit 401.

Further, the image generation unit 405 generates a pseudo three-dimensional image from the three-dimensional data extracted by the extraction unit 404 based on the specific result by the specific unit 402 and the specific result by the specific unit 403 (step ST304). At this time, the image generation unit 405 first sets the projection area by the projector 3 from the work area based on the specific result by the specific unit 402 and the specific result by the specific unit 403. Then, the image generation unit 405 transforms the three-dimensional data extracted by the extraction unit 404. At this time, the image generation unit 405 is stereoscopically viewed when the projected image is viewed from the position of the worker 11 specified by the specific unit 402 when the pseudo three-dimensional image is projected from the projector 3 into the set projection area. As such, the three-dimensional data is transformed. Then, the image generation unit 405 generates a pseudo three-dimensional image by converting the deformed three-dimensional data into a two-dimensional image. The process by the image generation unit 405 is performed every time the position of the worker 11 specified by the specific unit 402 changes.

Next, the control unit 406 controls at least one of the projection position and the projection angle of the projector 3 based on the specific result by the specific unit 402 and the specific result by the specific unit 403 (step ST305). At this time, the control unit 406 first sets the projection area by the projector 3 from the work area based on the specific result by the specific unit 402 and the specific result by the specific unit 403. The projection area set by the control unit 406 is the same as the projection area set by the image generation unit 405. Then, the control unit 406 controls at least one of the projection position and the projection angle of the projector 3 so that the projection can be performed on the set projection area. As a result, the projection position and the projection angle of the projector 3 are set. The process by the control unit 406 is performed every time the position of the worker 11 specified by the specific unit 402 changes.

Next, the output unit 407 outputs data indicating the pseudo three-dimensional image generated by the image generation unit 405 to the projector 3 (step ST306). After that, the projector 3 projects this pseudo three-dimensional image onto the work area. As a result, the projected image appears three-dimensionally to the operator 11. After that, the worker 11 performs the work on the product 12 according to the work instruction indicated by the projected image.

Next, the confirmation instruction unit 408 determines whether or not the work completion by the worker 11 has been confirmed (step ST307). If the confirmation instruction unit 408 has not confirmed the completion of the work in step ST307, the sequence returns to step ST307 and enters the standby state.

On the other hand, in step ST307, when the confirmation instruction unit 408 confirms the completion of the work, it determines whether the extraction unit 404 has extracted up to the three-dimensional data representing the last work process (step ST308).

In step ST308, when the extraction unit 404 determines that there is unextracted three-dimensional data, the confirmation instruction unit 408 instructs the extraction unit 404 to extract the next three-dimensional data (step ST309). .. After that, the sequence returns to step ST302. In step ST303, the extraction unit 404 extracts the three-dimensional data associated with the identification information indicating the next work process in response to the instruction from the confirmation instruction unit 408.

On the other hand, in step ST308, when the confirmation instruction unit 408 determines that all the three-dimensional data has been extracted by the extraction unit 404, the sequence ends.

Next, a specific example of the work instruction by the work support system according to the first embodiment will be shown.
In FIGS. 4 to 7, when the worker 11 places the box 16 in a designated place in the work area in a designated direction as the first work and tightens the four corners of the upper surface of the box 16 as the second work. show. In the following, as shown in FIG. 4A, a case where the worker 11 works on the work surface 14 placed in the work area is shown, and the product 12 is a part of the product 12 on the work surface 14 for positioning the box 16. It is assumed that the sheet metal part 15 is placed.

In this case, first, the work support system supports the first work.
Specifically, first, the identification unit 402 identifies the position of at least a part of the body including the eyes of the worker 11 from the image pickup result by the camera 1. Further, the identification unit 403 specifies the position and shape of the sheet metal component 15 from the image pickup result by the camera 2. In FIG. 4B, as shown by reference numeral 101, the image generation unit 405 and the control unit 406 use the projector 3 to set a region (two surfaces of the sheet metal component 15) where a part of the sheet metal component 15 is located from the work area. It is set in the projection area. Further, as shown in FIG. 5A, the extraction unit 404 extracts three-dimensional data indicating that the box 16 is placed in a designated place in the work area in a designated direction from the recording unit 401. The three-dimensional data shown in FIG. 5A is composed of a three-dimensional model representing the box 16 and a three-dimensional model with a downward arrow indicating that the box 16 is placed. Next, as shown in FIG. 5B, the image generation unit 405 transforms the three-dimensional data shown in FIG. 5A based on the specific result by the specific unit 402 and the specific result by the specific unit 403 to generate a pseudo three-dimensional image. do. Further, the control unit 406 controls at least one of the projection position and the projection angle of the projector 3 based on the specific result by the specific unit 402 and the specific result by the specific unit 403. Next, the output unit 407 outputs data indicating a pseudo three-dimensional image generated by the image generation unit 405 to the projector 3. After that, the projector 3 projects this pseudo three-dimensional image onto the work area. As a result, as shown by reference numeral 102 in FIG. 6A, the projected image appears three-dimensionally to the worker 11 so as to represent the work of placing the box 16 in the designated place in the work area in the designated direction. For example, as shown in FIG. 6B, when a third party other than the worker 11 views the projected image from a direction different from that of the worker 11, the projected image cannot be seen three-dimensionally. After that, the worker 11 places the box 16 in the designated direction in the designated place in the work area according to the work instruction indicated by the projected image.

Next, when the confirmation instruction unit 408 confirms the completion of the first work by the worker 11, the work support system supports the second work.
Specifically, first, the identification unit 402 identifies the position of at least a part of the body including the eyes of the worker 11 from the image pickup result by the camera 1. Further, the identification unit 403 specifies the position and shape of the sheet metal component 15 from the image pickup result by the camera 2. The image generation unit 405 and the control unit 406 project from the work area the area where a part of the sheet metal component 15 is located (a part of the two surfaces of the sheet metal component 15) and the area where the upper surface of the box 16 is located. Set to the projection area according to 3. Further, the extraction unit 404 extracts three-dimensional data indicating that the four corners of the upper surface of the box 16 are screwed from the recording unit 401. Next, the image generation unit 405 transforms the three-dimensional data based on the specific result by the specific unit 402 and the specific result by the specific unit 403 to generate a pseudo three-dimensional image. Further, the control unit 406 controls at least one of the projection position and the projection angle of the projector 3 based on the specific result by the specific unit 402 and the specific result by the specific unit 403. Next, the output unit 407 outputs data indicating a pseudo three-dimensional image generated by the image generation unit 405 to the projector 3. After that, the projector 3 projects this pseudo three-dimensional image onto the work area. As a result, as shown by reference numeral 103 in FIG. 7, the projected image appears three-dimensionally to the operator 11 as representing the work of screwing the four corners of the upper surface of the box 16. After that, the worker 11 tightens the screws on the four corners of the upper surface of the box 16 according to the work instructions indicated by the projected image.

Next, when the confirmation instruction unit 408 confirms the completion of the second work by the worker 11, the work support system ends the support.

In the above, the case where the image generation unit 405 and the control unit 406 set the projection area within the area where the sheet metal component 15 is located is shown. On the other hand, depending on the work content, the size of the component or the like may be large, and the projection area may extend beyond the area where the sheet metal component 15 is located. For example, when a large product such as a compressor is placed on a certain product 12, the projection area becomes large. In such a case, the image generation unit 405 and the control unit 406 set the projection region including the background region in addition to the region where the sheet metal component 15 is located.

Further, when the worker 11 is located in the blind spot of the work place, the work support system may project an image for notifying the work place. The case where the worker 11 is located in the blind spot of the work place is, for example, in FIG. 6A, the case where the worker 11 is located on the back surface of the sheet metal part 15.
Specifically, the image generation unit 405 notifies the work location when it is determined that the worker 11 is located in the blind spot of the work location based on the specific result by the specific unit 402 and the specific result by the specific unit 403. Generate an image to do. For example, when the worker 11 determines that the worker 11 is located on the back surface of the sheet metal component 15, the image generation unit 405 generates an image for notifying that the work location is the front surface of the sheet metal component 15. Then, the projector 3 projects the image generated by the image generation unit 405 onto a predetermined projection area. In the above example, the projector 3 projects the image onto the back surface of the sheet metal component 15.

As described above, in the work support system according to the first embodiment, the pseudo three-dimensional image is generated and the projector 3 is controlled based on the position of the worker 11 and the position and shape of the object existing in the work area, and the projector 3 is controlled. The pseudo three-dimensional image is projected from. As a result, the projected image can be seen three-dimensionally by the worker 11, the work instruction can be visually recognized by the worker 11, and the work can be easily performed by performing the work according to the work instruction indicated by the pseudo three-dimensional image. Can be carried out. Further, since the pseudo three-dimensional image is projected on the work area, the worker 11 does not have to take his eyes off the product 12 when confirming the work instruction, and the work time (tact time) can be expected to be shortened. Further, since characters are not used in the pseudo three-dimensional image, the worker 11 can easily perform the work regardless of the difference in language.

Further, the pseudo three-dimensional image can be viewed stereoscopically by the worker 11 with the naked eye, and the worker 11 does not need to wear a special device such as an HMD. Further, the pseudo three-dimensional image is a single image, and the work support system does not need to prepare a plurality of images as in the case of a three-dimensional image using binocular disparity.

Further, the identification unit 402 can specify the position of the worker 11 in real time from the image pickup result by the camera 1. Therefore, in the work support system, even if the worker 11 moves in the middle of the work, it is possible to generate a pseudo three-dimensional image by following the movement, and the worker 11 can always view the projected image in stereoscopic view.

In the above, when the specific unit 403 specifies the position and shape of an object existing in the work area from the image pickup result by the camera 2, and the image generation unit 405 and the control unit 406 perform processing based on the position and shape of the object. showed that. However, not limited to this, the recording unit 401 records information indicating the shape of the object that is assumed to exist in the work area in advance, and the specific unit 403 records the object existing in the work area from the image pickup result by the camera 2. Only the position of may be specified. In this case, the image generation unit 405 and the control unit 406 obtain the shape of the object existing in the work area from the recording unit 401.

As described above, according to the first embodiment, the camera 1 that captures the worker 11 working in the work area, the camera 2 that captures the work area, and the projector 3 that projects the input image onto the work area. A specific unit 402 that identifies the position of the worker 11 from the image capture result of the camera 1, a specific unit 403 that identifies the position of an object existing in the work area from the image image result of the camera 2, and an icon representing a work instruction. When the projected image is viewed from one direction from the three-dimensional data extracted by the extraction unit 404 based on the extraction unit 404 that extracts the dimensional data in order, the specific result by the specific unit 402, and the specific result by the specific unit 403. An image generation unit 405 that generates a pseudo-three-dimensional image that can be viewed stereoscopically, an output unit 407 that outputs data indicating a pseudo-three-dimensional image generated by the image generation unit 405 to the projector 3, and a worker 11 complete the work. It is provided with a confirmation instruction unit 408 that confirms and instructs the extraction unit 404 to extract the next three-dimensional data. As a result, in the work support system according to the first embodiment, it is possible to give a work instruction without characters while the worker 11 is looking at the work area without wearing a special device.

Embodiment 2.
In the first embodiment, the case where the image generation unit 405 generates a pseudo three-dimensional image which is a still image and the projector 3 projects the pseudo three-dimensional image on the work area is shown. On the other hand, the image generation unit 405 may generate a continuous animation of a plurality of pseudo three-dimensional images, and the projector 3 may project this animation on the work area. As a result, the worker 11 can more easily understand the contents of the work instruction.

Embodiment 3.
In the first and second embodiments, the image generation unit 405 and the control unit 406 do not specifically mention what kind of region the projection region is set to. On the other hand, when the projection area has a plurality of surfaces instead of a plane, the arithmetic unit 4 needs to perform an operation on the three-dimensional data for each area projected on each surface to generate a pseudo three-dimensional image. be. Therefore, it is desirable that the projection region is a region with as few irregularities as possible.

Therefore, the image generation unit 405 may detect a region having less unevenness from the specific result by the specific unit 403, set the region as the projection region by the projector 3, and generate a pseudo three-dimensional image. Similarly, the control unit 406 may detect a region having less unevenness from the specific result by the specific unit 403, set the region as the projection region by the projector 3, and control the projector 3. As a result, the arithmetic unit 4 can reduce the arithmetic load when generating a pseudo three-dimensional image.

Finally, with reference to FIG. 8, a hardware configuration example of the arithmetic unit 4 according to the first to third embodiments will be described.
Each function of the specific unit 402, the specific unit 403, the extraction unit 404, the image generation unit 405, the control unit 406, the output unit 407, and the confirmation instruction unit 408 in the arithmetic unit 4 is realized by the processing circuit 51. As shown in FIG. 8A, the processing circuit 51 may be dedicated hardware, or as shown in FIG. 8B, a CPU (Central Processing Unit, central processing unit,) that executes a program stored in the memory 53. It may be a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (also referred to as a Digital Signal Processor) 52.

When the processing circuit 51 is dedicated hardware, the processing circuit 51 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate). Array) or a combination of these is applicable. The functions of the specific unit 402, the specific unit 403, the extraction unit 404, the image generation unit 405, the control unit 406, the output unit 407, and the confirmation instruction unit 408 may be realized by the processing circuit 51, or the functions of each unit may be realized. It may be realized collectively by the processing circuit 51.

When the processing circuit 51 is the CPU 52, the functions of the specific unit 402, the specific unit 403, the extraction unit 404, the image generation unit 405, the control unit 406, the output unit 407, and the confirmation instruction unit 408 are software, firmware, or software and firmware. It is realized by the combination of. The software and firmware are described as a program and stored in the memory 53. The processing circuit 51 realizes the functions of each part by reading and executing the program stored in the memory 53. That is, the arithmetic unit 4 includes a memory 53 for storing a program in which, for example, each step shown in FIG. 3 is eventually executed when executed by the processing circuit 51. Further, it can be said that these programs cause the computer to execute the procedures and methods of the specific unit 402, the specific unit 403, the extraction unit 404, the image generation unit 405, the control unit 406, the output unit 407, and the confirmation instruction unit 408. .. Here, examples of the memory 53 include non-volatile or volatile semiconductor memories such as RAM, ROM, flash memory, EPROM, and EEPROM, magnetic disks, flexible disks, optical disks, compact disks, mini disks, DVDs, and the like. do.

For each function of the specific unit 402, the specific unit 403, the extraction unit 404, the image generation unit 405, the control unit 406, the output unit 407, and the confirmation instruction unit 408, some of them are realized by dedicated hardware, and some of them are realized. It may be realized by software or firmware. For example, the specific unit 402 is realized by the processing circuit 51 as dedicated hardware, and the specific unit 403, the extraction unit 404, the image generation unit 405, the control unit 406, the output unit 407, and the confirmation instruction unit 408 are realized. The function can be realized by the processing circuit 51 reading and executing the program stored in the memory 53.

As described above, the processing circuit 51 can realize each of the above-mentioned functions by hardware, software, firmware, or a combination thereof.

In the present invention, within the scope of the invention, it is possible to freely combine each embodiment, modify any component of each embodiment, or omit any component in each embodiment. be.

The work support system according to the present invention can be used for a work support system or the like that enables work instructions without characters while the worker is looking at the work area without wearing a special device. Suitable for.

1 camera (1st image pickup device), 2 camera (2nd image pickup device), 3 projector (projection device), 4 arithmetic unit, 11 workers, 12 products, 13 trolleys, 14 work surfaces, 15 sheet metal parts, 16 boxes, 51 processing circuit, 52 CPU, 53 memory, 401 recording unit, 402 specific unit (first specific unit), 403 specific unit (second specific unit), 404 extraction unit, 405 image generation unit, 406 control unit, 407 output unit , 408 Confirmation indicator.

Claims (9)

A first imaging device that captures images of workers working in the work area,
A second imaging device that captures the work area,
A projection device that projects the input image onto the work area,
The first specific unit that identifies the position of the worker from the image pickup result by the first image pickup device, and
A second specific unit that identifies the position of an object existing in the work area from the image pickup result by the second image pickup device, and
An extraction unit that extracts 3D data, which is an icon representing work instructions, in order,
Pseudo-three-dimensional image that can be viewed stereoscopically when the projected image is viewed from one direction from the three-dimensional data extracted by the extraction unit based on the specific result by the first specific unit and the specific result by the second specific unit. An image generator that generates an image and
An output unit that outputs data indicating a pseudo three-dimensional image generated by the image generation unit to the projection device, and an output unit.
A work support system including a confirmation instruction unit that confirms the completion of work by an operator and instructs the extraction unit to extract the next three-dimensional data. The work support system according to claim 1, wherein the image generation unit generates a pseudo three-dimensional image each time the position of the worker specified by the first specific unit changes. The work support system according to claim 1, wherein the first specific unit specifies at least the position of a part of the body including the eyes of the worker as the position of the worker. The work support system according to claim 1, wherein the image generation unit generates a continuous animation of a plurality of pseudo three-dimensional images. The image generation unit is characterized in that a region having less unevenness is detected from the specific result by the second specific unit, the region is set as a projection region by the projection device, and a pseudo three-dimensional image is generated. The work support system according to claim 1. The first imaging device according to claim 1, wherein at least one of the imaging position and the imaging angle is variably configured so that the region where the operator is supposed to be located is included in the imaging region. Work support system. In the projection device, at least one of the projection position and the projection angle is variable.
The claim is characterized in that it includes a control unit that controls at least one of the projection position and the projection angle of the projection device based on the specific result by the first specific unit and the specific result by the second specific unit. The work support system described in 1. The seventh aspect of claim 7, wherein the control unit controls at least one of the projection position and the projection angle of the projection device each time the position of the worker specified by the first specific unit changes. Work support system. The claim is characterized in that the control unit detects a region having less unevenness from a specific result by the second specific unit, sets the region as a projection region by the projection device, and controls the projection device. The work support system described in 7.

Images