JP2014071756A - Work assistance system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a work assistance system and a program, which are capable of detecting a work target article and instructing movement from the present azimuth of a worker relative to the work target article to an azimuth for working thereon.SOLUTION: A CPU executes: detecting a feature region from a photographed image that has taken a work target article (S13) to determine the present azimuth of a worker relative to the work target article (S19); obtaining a work azimuth of a next target region of the work target article (S21) to calculate a shortest moving angle for moving the worker from the present azimuth to the work azimuth (S35); displaying an arrow image of a length in accordance with the moving angle at a position within a display image that is set relatively with the position of the work target article within the photographed image and in accordance with a direction of the arrow pointing (S37); and displaying an image based on procedure information showing a work procedure for the target region (S27) when the present azimuth becomes the work azimuth as a result of movement of the worker (yes for S25).

Description

  The present invention relates to a work assistance system and a program for presenting work instructions to a display device mounted on a worker's head.

  A system is known in which an operator (inspector) can display information on an object to be worked (inspection object) on a head-mounted display (head-mounted display device) attached to the head of the operator. (See, for example, Patent Document 1). In Patent Document 1, an inspector goes around the plant and inspects an inspection object. An ID tag is attached at a position corresponding to the inspection target, an identification signal emitted from the ID tag is read, and information on the inspection target corresponding to the identification signal is displayed on the head mount display of the inspector.

Japanese Patent Laid-Open No. 10-214038

  However, in Patent Document 1, it is necessary for the inspector to grasp the route of the tour in advance, and if the route is incorrect, there is a problem in that it cannot reach the position to be inspected and cannot perform the work. In addition, it is necessary to attach an ID tag at a position corresponding to an inspection target in advance, which is troublesome. In addition, when there are multiple inspection points in one inspection object, an ID tag will be attached to each inspection point. However, especially where there are multiple inspection points in the field of view, which inspection point corresponds to the identification signal. There was also a problem of not knowing whether or not.

  The present invention has been made to solve the above-described problems, and can detect a work object and instruct movement from the current direction of the worker to the work direction with respect to the work object. An object is to provide a work support system and program.

  According to an embodiment of the present invention, a display device configured to be attachable to a worker's head and capable of displaying an image, a photographing device capable of photographing a work object on which the worker works, and the work First acquisition means for acquiring procedure information for displaying a procedure of an operation to be performed on the work object on the display device as an image by a person facing the work object from a predetermined direction with respect to the work object And, based on the specific information, second acquisition means for acquiring a predetermined image that is an image of a predetermined part in the work object from the captured image captured by the imaging device as specific information for specifying the orientation, When the determination means for determining the current direction of the worker with respect to the work object and the current direction of the worker are the predetermined direction targeted in the work procedure indicated by the procedure information, the display device In First display means for displaying an image based on the procedure information, and when the worker's current orientation is not the predetermined orientation targeted in the work procedure indicated by the procedure information, the worker's current orientation And a second display means for displaying on the display device instruction information determined according to the relationship between the predetermined orientation and the predetermined orientation.

  In this embodiment, the operator can be instructed to move from the current direction to the predetermined direction for the predetermined part to perform the next operation, so that the operator can change the order of the operations, It will not move in a different direction. When the operator finishes the work on the predetermined part, the operator does not need to confirm the moving route to the next predetermined part by using a procedure manual or the like, and can immediately move to the work on the next predetermined part. Work can be done.

It is a perspective view which shows the external appearance of HMD1. It is a block diagram which shows the electric constitution of HMD1. It is a flowchart which shows a work assistance program. It is a figure which shows the example of the picked-up image 350 which image | photographed the work target object 400. FIG. It is a figure which shows a mode that the image 470 which shows instruction information with the real image of the work target object 400 is displayed. It is a figure which shows the example of the picked-up image 360 which image | photographed the work target object 400. FIG. It is a figure which shows a mode that the procedure image 480 is displayed with the real image of the work target object 400. FIG. It is a figure which shows a mode that the image 490 which shows instruction information with the real image of the work target object 400 is displayed. It is a figure which shows a mode that the procedure image 500 is displayed with the real image of the work target object 400. FIG.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The drawings to be referred to are used to explain technical features that can be adopted by the present invention. The configuration of the illustrated apparatus is not intended to be limited only to the form, but merely an illustrative example.

  The work assistance system according to the present invention includes a head mounted display 1 (hereinafter referred to as “HMD1”, see FIG. 1) and a personal computer 300 (hereinafter, referred to as “PC300”, see FIG. 2) that transmits information related to work to the HMD1. It is a system composed of The HMD 1 displays an image so as to be superimposed on an external scene that passes through a half mirror 8 (described later) disposed in front of the operator's eyes. The HMD 1 is connected to the PC 300 using known wireless communication, and transmits / receives various data including image data to / from the PC 300. In the following description, the upper, lower, right diagonally downward, left diagonally upward, right diagonally upward and left diagonally downward of FIG. 1 are the upper, lower, forward, backward, right and left sides of the HMD 1, respectively.

  As shown in FIG. 1, the HMD 1 includes a projection device 100 and a control device 200. The projection apparatus 100 is used by being mounted on spectacles 91 that are dedicated mounting tools. Projection apparatus 100 may be attached to other wearing tools such as glasses, helmets, and headphones that are used daily by workers. The projection apparatus 100 irradiates the operator's eyes with image light. The projection device 100 is detachably connected to the control device 200 via a harness 150. The control device 200 is used by being mounted on, for example, an operator's waist belt. The control device 200 controls the projection device 100.

  The configuration of the projection apparatus 100 will be described. The projection apparatus 100 includes a housing 2. The housing 2 includes an image display unit 36 (see FIG. 2) and an eyepiece optical system (not shown). The image display unit 36 displays an image based on a video signal transmitted from the control device 200 via the harness 150. The image display unit 36 is, for example, a spatial modulation element such as a liquid crystal element and a light source. The image display unit 36 may be a retinal scanning display unit including a two-dimensional optical scanner and a light source, an organic EL (Organic Electro-luminescence) element, or the like. The eyepiece optical system condenses image light indicating an image displayed on the image display unit 36 so as to guide it to the eyes of the operator. At least a part (for example, half) of the image light emitted from the eyepiece optical system is reflected by the half mirror 8 provided on the left side of the housing 2 and enters the left eyeball (not shown) of the operator. Since the half mirror 8 transmits at least part of the light from the real image in the outside world, the operator can visually recognize the image superimposed on the real image (outside landscape) in his field of view.

  The glasses 91 are configured to hold the projection device 100 on the operator's head. The housing 2 of the projection apparatus 100 is attached to the glasses 91 by a support unit 96. The support portion 96 is provided on the upper left end side (upper right end in FIG. 1) of the glasses 91 when viewed from the operator side. An adjustment rod 98 is assembled to the support portion 96. The adjustment rod 98 can slide in the left-right direction with respect to the support portion 96. The adjustment bar 98 extends in the vertical direction, and has a triangular protrusion 97 at the upper end. The adjustment rod 98 protrudes downward from the support portion 96 with the protrusion 97 exposed on the upper surface side of the support portion 96. The housing 2 has a mounting portion 49 at a portion facing the glasses 91 (on the back side). The attachment portion 49 is attached to the adjustment rod 98. The housing 2 can be moved in the vertical direction along the adjustment rod 98. Further, the housing 2 can be moved in the left-right direction with respect to the support portion 96 together with the adjustment rod 98 by sliding the protrusion 97 to the left and right. The operator can place the half mirror 8 at a position that matches the position of the eyeball.

  The eyeglasses 91 include a camera 39 at the approximate center in the left-right direction of the frame 94. The camera 39 captures an image of an external landscape in front of the worker's field of view. The shooting direction and angle of view of the camera 39 are set in advance according to the field of view of the operator. More specifically, the shooting direction and the angle of view of the camera 39 are the same as the display range of image light from the image display unit 36 in which the shooting range of the external scenery by the camera 39 can be superimposed on the real image that passes through the half mirror 8. It is set to almost match. In addition, the glasses 91 include a headset 35 incorporating a bone conduction earphone and a microphone on the right end side of the frame 94. Since the shape of the glasses 91 itself is similar to that of normal glasses, detailed description is omitted.

  The configuration of the control device 200 will be described. The control device 200 is a substantially rectangular parallelepiped system box, for example. The control device 200 includes an operation unit 50 including a power switch 51 that incorporates a power lamp 52. An operator can operate the power switch 51 to turn on or off the power of the HMD 1. In addition, the operator can perform various settings in the projection apparatus 100, various operations during use, and the like via the operation unit 50. The control device 200 can be connected to the PC 300 via wireless communication and can transmit and receive various data. The control device 200 includes a wired communication interface, and may be connected to the network 250 using a communication cable, or may be directly connected to the PC 300. The PC 300 described later is a personal computer, but may be other devices such as a smartphone or a network-compatible storage.

  The electrical configuration of the HMD 1 will be described with reference to FIG. The projection apparatus 100 includes a CPU 21 that controls the entire projection apparatus 100. The CPU 21 is electrically connected to the program ROM 23, RAM 25, image display unit 36, interface 12, and connection controller 37. The CPU 21 is electrically connected to the camera 39 and the headset 35 via the interface 12. The program ROM 23 stores various programs executed by the CPU 21. Various programs are stored in the program ROM 23 when the projection apparatus 100 is shipped. The CPU 21 can also execute a program stored in a flash ROM 47 of the control device 200 described later. The RAM 25 temporarily stores various data.

  As described above, the image display unit 36 displays an image based on the video signal. Hereinafter, an image displayed on the image display unit 36 is referred to as a “display image”. The interface 12 is connected to the camera 39 and the headset 35 and controls signal input / output. The connection controller 37 is connected to the connection controller 48 of the control device 200 via the harness 150 and performs wired communication. The camera 39 captures a still image or a moving image (hereinafter referred to as “captured image”). The headset 35 includes a bone electric earphone and a microphone. Note that the projector 100 or the control device 200 may incorporate a speaker and a microphone.

  The electrical configuration of the control device 200 will be described. The control device 200 includes a CPU 41 that controls the entire control device 200. The CPU 41 is electrically connected to the program ROM 43, RAM 45, flash ROM 47, connection controller 48, video RAM 57, image processing unit 59, interface 55, and wireless communication unit 53. The program ROM 43 stores various programs executed by the CPU 41. Various programs are stored in the program ROM 43 when the control device 200 is shipped. The CPU 41 can also execute a program stored in the flash ROM 47.

  The RAM 45 temporarily stores various data. The flash ROM 47 stores various programs installed after shipment of the control device 200 and various data used in a work assistance program described later (sample images for searching for characteristic parts, object identification table for identifying work objects. Etc. (described later). The flash ROM 47 stores the procedure information when procedure information (described later) is acquired from the PC 300 described later. The connection controller 48 is connected to the connection controller 37 of the projection apparatus 100 via the harness 150 and performs wired communication. The wireless communication unit 53 wirelessly connects to an access point (not shown) of the network 250 and communicates with other devices connected to the network 250. The wireless communication unit 53 may directly connect to the PC 300 wirelessly without going through the network 250. The interface 55 is connected to an operation unit 50 including a power switch 51 and a power lamp 52, and inputs and outputs operation input signals, lamp lighting signals, and the like by an operator. The image processing unit 59 performs processing for forming a display image to be displayed on the image display unit 36 of the projection device 100. The video RAM 57 forms a virtual screen in which the display image is virtually displayed in the storage area in order to generate a video signal for displaying the display image formed by the image processing unit 59 on the image display unit 36.

  The PC 300 is a known personal computer equipped with a CPU, ROM, RAM, and the like. In the present embodiment, the PC 300 holds procedure information (described later) regarding the procedure of work performed on a work object for a plurality of work objects, and accumulates (stores) it in a hard disk drive (HDD) 350 as a database. The PC 300 functions as a server that provides procedure information via the network 250 in response to a request from the CPU 41 of the control device 200. Although not shown, the PC 300 includes a communication unit for connecting to the network 250.

  Each process of the work assistance program executed by the CPU 41 of the control device 200 of the HMD 1 will be described with reference to FIGS. First, the contents of various processes performed by the CPU 41 as the work assistance program is executed will be described. In processing accompanying execution of a work assistance program to be described later, the CPU 41 analyzes a photographed image photographed by the camera 39. The CPU 41 analyzes the photographed image and detects a “feature image” that is an image of a part having a feature amount in the work target imaged in the photographed image.

  For example, when the work target is a vehicle, the “part having the characteristic amount” is a part having a unique shape different from other vehicles such as an emblem, a logo mark of a car name, a tail lamp, an overall shape, a characteristic Such as painting. That is, when a captured image of a target vehicle is analyzed, the vehicle type of the vehicle, a side surface of a side surface of the vehicle that appears in the captured image, and the like can be specified. Hereinafter, a part having a feature amount is referred to as a “feature part”. A mark such as a marker may be provided in advance on the work object to make it a characteristic part.

  The details of the image analysis are well known and will not be described here. For example, feature parts are detected using various known methods such as detection of contour lines, matching of feature parts with sample images, and detection of color schemes. Is possible. In the present embodiment, it is assumed that the CPU 41 detects a feature image that is an image of a feature part in a captured image by matching with a sample image. The “sample image” is an image in which a part to be detected as a characteristic part of the work object is captured or drawn in advance. The sample image is an image obtained when the feature part is viewed in a direction orthogonal to the side surface on which the feature part of the work object can be confirmed. A plurality of sample images are prepared for each type of work object, and are stored in the flash ROM 47 in advance. Further, a table (hereinafter referred to as “object identification table”) in which each work object is associated with a characteristic part of each work object is also created and stored in the flash ROM 47 in advance.

  The CPU 41 searches a portion similar to the sample image in the captured image while performing rotation, enlargement / reduction, deformation, etc. of the sample image. When the CPU 41 finds a similar part, that is, a characteristic part, the CPU 41 stores a characteristic image, which is an image of the characteristic part, in the RAM 45 as characteristic information indicating the direction. “Feature information” is information that the feature image represents in the captured image. For example, the position (coordinates) of the feature image in the captured image, the rotation angle of the feature image with respect to the sample image, the enlargement / reduction rate, the deformation rate, etc. It is. The feature information also includes information on association between the searched feature image and the sample image of the search source.

Further, the CPU 41 identifies a work object that appears in the photographed image in the process of the work assistance program. At that time, the CPU 41 refers to the object identification table. Further, in the object identification table, when the work object is viewed from a plurality of reference directions (for example, six surfaces (plane, bottom surface, front surface, back surface, and left and right side surfaces)), it can be confirmed on each surface. Information such as the positional relationship, size, and orientation of the characteristic parts is associated with each work object.

As described above, the feature information includes the position (coordinates) of the feature image in the captured image, the rotation angle of the feature image with respect to the sample image, the enlargement / reduction rate, the deformation rate, and the like. Therefore, the CPU 41 can specify the side surface of the work object that can confirm the feature part that appears in the detected feature image, based on the position of the feature image in the captured image. Further, the CPU 41 may specify from which direction the side of the specified work object is facing the side surface based on the rotation angle, the enlargement / reduction ratio, the deformation rate, etc. of the feature image with respect to the sample image. it can. Therefore, the CPU 41 can specify from which direction (direction) the captured image is captured with respect to the work object.

  In addition, the CPU 41 acquires procedure information corresponding to the identified work object in the process of the work assistance program. “Procedure information” is information that can indicate the procedure of the work performed on the work object by an image (including superimposed characters and symbols). Hereinafter, an image based on the procedure information is referred to as a “procedure image”. Since the procedure image indicates a part (hereinafter, referred to as “target part”) that is a target to be worked on in the work target, work is performed from a predetermined direction (hereinafter, referred to as “working direction”) desirable for the work. It includes an image in which the object is drawn or photographed, an image showing the target region, characters / symbols, and the like. The procedure image may be a still image or a moving image. Note that the work direction is a direction indicated by an angle when the worker faces the work object from the front as a reference azimuth of the azimuth and moves radially outward of the work object in the circumferential direction. Is set. Further, the procedure information includes information on the work direction, and when the work target has a plurality of target parts, information on the work direction corresponding to each target part. Further, the procedure information includes information on the total number of target parts and information on the order in which the work is performed on each target part and the order set in advance.

  Further, the CPU 41 presents instruction information to the worker in the processing of the work assistance program. “Instruction information” is information determined according to the relationship between the current orientation of the worker (the orientation of the worker with respect to the work object when the photographed image is taken) and the work orientation. Specifically, This is information indicating an instruction to move the worker from the current direction to the work direction. The instruction information is, for example, image information representing an image of an arrow indicating a direction from the current direction to the work direction. In addition, the instruction information is, for example, image information that expresses, in characters, contents that inform the worker of the direction from the current direction to the work direction. Alternatively, the instruction information is, for example, voice information that informs the worker by voice the direction from the current direction to the work direction.

  Various operations performed by the CPU 41 in accordance with the execution of the work assistance program will be described. When the operator turns on the power switch 51 of the HMD 1, the CPU 41 executes a work assistance program stored in the program ROM 43 shown in FIG. 3. In the initial setting at the start of execution of the work assistance program, the CPU 41 initializes the flag and data stored in the RAM 45, operates the wireless communication unit 53, and connects to the network 250 by wireless communication (S9). The CPU 41 tries to connect with the PC 300 disclosed in the network 250 in advance, and confirms whether or not it is possible to log in with a preset ID and password.

  The camera 39 of the projection apparatus 100 always shoots the scenery of the outside world, and the CPU 21 stores the captured image while overwriting the RAM 25. The CPU 41 outputs an instruction signal to the CPU 21, acquires the latest captured image from the RAM 25, and stores it in the RAM 45 (S11). In the acquired captured image, the CPU 41 performs matching with the sample image as described above, searches for a characteristic part, and detects a characteristic image (S13). If the feature image cannot be detected, the CPU 41 determines that the work object is not captured in the captured image (S15: NO), returns to S11, and the operator moves to a position where the work object is captured in the captured image. Repeat steps S11 to S15.

  When the feature image can be detected, the CPU 41 stores the detected feature image in the RAM 45 as feature information indicating the orientation. The CPU 41 identifies a work object that appears in the captured image based on the captured image and the feature information (S15: YES). The CPU 41 searches for a work object having a feature part detected as a feature image from an object identification table stored in the flash ROM 47. If CPU41 specifies a work target in a target object identification table, it will memorize | store an identification result in RAM45.

  In order to facilitate the explanation of the orientation with respect to the work object, a dice-shaped object whose side surface can be easily identified is taken as an example of the work object 400. FIG. 4 shows how the work object 400 appears in the captured image 350. The captured image 350 includes a first side surface 410 having a red eye 405 having a different size from the other eyes, a second side surface 420 having two black eyes 406, and a black side surface. A third side surface 430 having three eyes 406 is shown. The red eyes 405 and the black eyes 406 are characteristic portions. In the captured image 350, the CPU 41 matches the sample image with one red eye 405, two black eyes 406 having the same deformation rate, and the two previous black eyes 406 with different deformation rates. Three black eyes 406 having the same deformation rate are detected as characteristic portions. The CPU 41 refers to the object identification table and identifies that the object having the characteristic part is the work object 400.

  As shown in FIG. 3, the CPU 41 connects to the PC 300 via the network 250 and requests procedure information corresponding to the identified work object. The CPU 41 stores the procedure information acquired from the PC 300 in the flash ROM 47 (S17). Further, the CPU 41 stores in the RAM 45 information on the target part included in the procedure information (information on the work direction, information on the total number, information on the order in which work is performed on each target part). When the information about the target part corresponding to the procedure information of the work target identified this time is already stored in the RAM 45, that is, when the process of S17 is not performed for the first time, the CPU 41 does not overwrite the information about the target part. CPU41 can acquire the procedure information corresponding to a work target object by identifying a work target object in S15. Therefore, it is possible to save labor for the operator to prepare procedure information corresponding to the work object in advance.

  Based on the photographed image and the feature information, the CPU 41 determines from which direction the worker is currently directed to the work object shown in the photographed image (S19). The CPU 41 determines the orientation (current orientation) of the worker with respect to the work target when the photographed image is captured based on the reference orientation described above.

  For the work object 400 in FIG. 4, the CPU 41 determines that the feature image of the red eye 405 has a larger enlargement ratio and a lower deformation ratio than the other eyes (black eyes 406) based on the feature information. I understand. Therefore, the CPU 41 positions the first side surface 410 having the red eyes 405 closer to the operator than the second side surface 420 having the two black eyes 406 and the third side surface 430 having the three black eyes 406. And it turns out that it is a side which faces the operator side. That is, the CPU 41 detects that the current direction of the worker with respect to the work object 400 is a direction in which the worker faces the first side surface 410. More specifically, when the first side surface 410 is the front surface (reference) of the work object 400, the CPU 41 has a deformation rate of the red eye 405 on the first side surface 410 and a black eye on the second side surface 420 that is the right side surface. Based on the deformation rate of 406, the current orientation is obtained.

  As shown in FIG. 3, the CPU 41 next refers to the information on the target part stored in the RAM 45 in S <b> 17, and the work direction of the target part to be worked next in the work order is the target part that has not been completed. Is acquired (S21). The CPU 41 analyzes the photographed image and specifies the position (coordinates) of the work object in the photographed image, for example, by detecting the outline of the work object (S23).

  The CPU 41 confirms whether or not the current orientation of the worker determined in S19 is an orientation included in a predetermined angle range centered on the work orientation of the target part acquired in S21 (S25). In other words, when the worker faces the work object from the current direction, the CPU 41 confirms whether or not the position of the target part where the work is to be performed is a desired position for the work.

  The CPU 41 displays a procedure image on the image display unit 36 when the current azimuth is not included in the predetermined angle range of the work azimuth and there is no target part at the desired position of the work target in the current azimuth (S25: NO). If yes, the procedure image is hidden (S33). CPU41 calculates | requires the shortest moving distance when an operator moves to the work direction of the next object part from the present direction (S35). Specifically, the CPU 41 calculates the angle difference between the worker's current direction and the work direction, and determines the smaller value as the movement angle (the minimum movement angle in the circumferential direction of the work object).

  The CPU 41 presents instruction information for moving the worker from the current direction to the work direction of the next target part (S37). In the image processing unit 59, the CPU 41 indicates the direction from the current direction to the working direction in the shortest direction, and forms an arrow image having a length corresponding to the movement angle as instruction information. In addition, since the CPU 41 displays the image of the arrow at a position in the display image that is set in advance relative to the work object in accordance with the direction indicated by the arrow, the CPU 41 identifies the work object specified in the captured image in S23. Based on the position, the arrangement position of the arrow image in the display image is determined. Further, the CPU 41 forms instruction information represented by characters (sentences) or the like prepared in advance according to the direction indicated by the arrow, and arranges it in the display image. The display image formed by the image processing unit 59 is formed as a virtual screen in the video RAM 57, and the CPU 41 generates a video signal based on the virtual screen and outputs the video signal to the projection device 100. The image display unit 36 of the projection device 100 displays a display image based on the video signal. Further, the CPU 41 outputs an audio signal to the headset 35 of the projection apparatus 100 based on voice instruction information prepared in advance according to the direction indicated by the arrow. The operator recognizes the instruction information shown in the display image incident on the eyeball via the half mirror 8 and the voice uttered from the bone conduction earphone of the headset 35. The CPU 41 indicates to the worker the direction of the shortest direction toward the work direction with an arrow, that is, guides the worker along the minimum path toward the work direction, so that the worker can quickly move to the next target part. Can be done. The CPU 41 can indicate to the worker the direction of movement to the next target site with an arrow having a length corresponding to the difference (movement angle) between the current direction and the work direction. The movement direction and distance can be grasped, and movement to the next target part can be performed quickly. By displaying the image of the arrow at a relative position according to the moving direction with respect to the work object in the display screen, the worker can locate the work object at any position in the display screen. The moving direction can be intuitively grasped with respect to the work object, and the movement to the next target part can be quickly performed.

  In FIG. 4, it is assumed that the next target portion of the work object 400 is on the sixth side surface 460 (see FIG. 6) on the back side of the first side surface 410. As shown in FIG. 5, an image 470 indicating instruction information is displayed on the right hand side of the real image of the work object 400 that passes through the half mirror 8. The image 470 indicating the instruction information includes an arrow image 471 that instructs the work object 400 to move to the back side in a clockwise direction, and a character image 472 that constitutes an explanatory note of the movement below the image 470. Also, voice instruction information is presented. The worker moves around the work object 400 clockwise to a position facing the sixth side surface 460 in accordance with the presented instruction information.

  As shown in FIG. 3, the CPU 41 returns to S11 after the process of S37. During the movement of the worker, since the current azimuth is not included in the predetermined range of the work azimuth, the CPU 41 repeats S11 to S25 and S33 to S37 until the worker reaches the work azimuth.

  A captured image 360 shown in FIG. 6 shows a sixth side surface 460 having six black eyes 406, a second side surface 420 having two black eyes 406, and a third side surface 430 having three black eyes 406. As described above, the CPU 41 detects, based on the feature information of the work object 400, that the worker is in an orientation that faces the sixth side surface 460.

  As shown in FIG. 3, when the worker reaches the work direction and the current position of the worker is within a predetermined angle range of the work direction (S25: YES), the CPU 41 displays the image display unit 36. Then, a procedure image showing the procedure of the work performed on the target part is displayed (S27). The CPU 41 displays the procedure image in the display image while avoiding the position of the work object specified in the captured image.

  FIG. 7 shows a state in which the procedure image 480 is displayed on the right-hand side of the real image so as not to overlap the real image of the work object 400 that passes through the half mirror 8. The procedure image 480 includes an image 481 indicating the target part of the sixth side surface 460 of the work object 400 and a character image 482 constituting an explanatory text indicating the work procedure at the lower part thereof. The worker performs work on the target part according to the displayed procedure image 480.

  The CPU 41 detects the completion of the work on the target part by, for example, matching with a completed image of the target part acquired in advance as procedure information. The CPU 41 continues displaying the procedure image and returns to S27 (S29: NO) until the work on the target part is not completed. When the CPU 41 detects the completion of the work on the target part (S29: YES), the CPU 41 stores a flag indicating the work completion for the target part for which the work has been completed among the information on the target part stored in the RAM 45, and sets the total number of target parts to 1 cut back. The CPU 41 proceeds to S31, and if the total number of target parts is 1 or more, the CPU 41 determines that there is a work that has not been completed yet (S31: YES), and returns to S11. The CPU 41 can detect the procedure information of the next target part by detecting the completion of the work on the target part without depending on the operator's operation, so that the operator's hand is not bothered. The worker can work efficiently.

  In FIG. 7, it is assumed that the next target portion of the work target object 400 is on the second side surface 420. As shown in FIG. 8, an image 490 indicating instruction information is displayed on the left hand side of the real image of the work object 400 that passes through the half mirror 8. The image 490 indicating the instruction information includes an arrow image 491 for instructing to move the work object 400 to the left side in a counterclockwise direction, and a character image 492 constituting a description of the movement at the lower part. Also, voice instruction information is presented. In the case of FIG. 5 described above, since the work object 400 moves from the front side to the back side, the movement angle is about 180 degrees. In the case of FIG. 7, the work object 400 moves from the front side to the left side. Therefore, the movement angle is approximately 90 degrees. Therefore, when forming an arrow image as instruction information, the CPU 41 uses the length of the arrow shown in the arrow image 491 at the current movement as the length of the arrow shown in the arrow image 471 at the previous movement. Make it shorter. The worker moves around the work object 400 counterclockwise to a position facing the second side surface 420 according to the presented instruction information.

  As shown in FIG. 9, the procedure image 500 including the image 501 indicating the target part of the second side surface 420 and the character image 502 is displayed on the right hand side of the real image of the work object 400 that passes through the half mirror 8 as described above. Is done. The worker performs work on the target part according to the procedure image 500.

  As shown in FIG. 3, when the total number of target parts is reduced by 1 with the completion of work in S29 and becomes 0, the CPU 41 determines that all work for the work target is completed in S31 (S31: NO). The work assistance program is terminated.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention. The image analysis in S13 is performed by the CPU 41 of the control device 200, but the CPU 41 transmits the captured image to the PC 300 via the network 250, the CPU of the PC 300 analyzes the captured image, and transmits the analysis result to the CPU 41. May be.

  The CPU 41 presents the instruction information so that the movement distance is the shortest. However, when an obstacle or the like appears on the movement route in the photographed image, the instruction information indicating the shortest route while avoiding the obstacle is presented. Also good. Although an arrow image is displayed as the instruction information, the arrow image may be shown as a still image, or may be shown as a moving image using an animation or the like that blinks or the position of the image moves within the display screen. In this embodiment, in addition to an arrow image, a character image is displayed and a voice instruction is uttered. However, only an arrow image may be displayed and a character or voice instruction may be omitted. Or, there is no arrow image, and an instruction by text or voice may be presented. Further, the CPU 41 displays the image of the arrow at a position relative to the work object in the display image according to the direction indicated by the arrow, but the absolute image in the display image according to the direction indicated by the arrow. You may display in a position. For example, if a clockwise movement is instructed, an arrow image indicating a clockwise movement may be displayed at the right end of the display image. By displaying an arrow image at an absolute position corresponding to the moving direction in the display screen, the operator can intuitively grasp the moving direction and quickly move to the next target part. be able to.

  In S <b> 29, the CPU 41 determines the completion of the operation by matching with the completed image of the target part, but may determine that the operation is completed based on the operation of the operation unit 50 by the operator. Alternatively, a timer may be set, and the CPU 41 may determine that the work has been completed when a preset time has elapsed for the work in the target region. Alternatively, the CPU 41 may determine that the work has been completed when the photographed image is analyzed and it is detected that the tool used for the work has been placed in a predetermined place.

  Instead of the camera 39, a fixed-point camera that can photograph the surrounding state including the work object may be installed at a predetermined fixed position. The PC 300 may generate instruction information based on the captured image of the fixed point camera and transmit the instruction information to the HMD 1. Further, although the CPU 41 has acquired the instruction information from the PC 300 via the network 250, it may be stored in advance in the flash ROM 47 of the control device 200 and acquired from the flash ROM 47. In this case, the HMD 1 may not be connected to the network 250. That is, the HMD 1 alone may be able to instruct the worker to move to the work direction.

  In the present embodiment, as the instruction information, information on a form in which the worker is guided along the route from the current direction to the work direction and the worker is moved with respect to the work target is presented. May be presented as information in a form in which the work object is moved to direct the work direction of the work object to the current direction. For example, when the table on which the work object is placed is manually rotated or moved, information indicating in which direction and how much the table is rotated or moved may be presented.

  In the present embodiment, the current orientation of the worker with respect to the work target is determined based on the feature information (information such as the positional relationship of the feature image, the enlargement / reduction ratio, and the deformation rate) of the feature image. For example, the feature image of the feature part may be stored in the flash ROM 47 in association with the orientation with respect to the work target. In this case, if the CPU 41 detects a feature image in the captured image, the CPU 41 may determine the orientation corresponding to the feature image as the current orientation of the worker. Alternatively, a sample image obtained by capturing the entire image of the work object from a plurality of directions may be captured in advance and stored in the flash ROM 47. In this case, since the background or the like of the work object shown in the photographed image may be different from the sample image, the CPU 41 may specify the sample image most similar to the photographed image and determine the current orientation of the worker.

  In the present embodiment, the image display unit 36 corresponds to the “display device” of the present invention. The camera 39 corresponds to a “photographing device”. The work direction corresponds to a “predetermined direction”. CPU41 which acquires procedure information in S17 is equivalent to a "first acquisition means". The characteristic part corresponds to a “predetermined part”. The feature image corresponds to a “predetermined image”. The feature information corresponds to “specific information”. In S13, the CPU 41 that analyzes the captured image to detect the feature image and obtains the feature information corresponds to the “second obtaining unit”. The CPU 41 that determines the current orientation of the worker with respect to the work target based on the feature information acquired by analyzing the captured image in S19 corresponds to “determination means”. The CPU 41 that displays the procedure image in S27 corresponds to “first display means”. CPU41 which displays the image of the arrow based on instruction information in S37 corresponds to a "second display means". The half mirror 8 corresponds to a “display unit”. The CPU 41 that specifies the position of the work object in the photographed image in S23 corresponds to “specifying means”. The CPU 41 that identifies the work object based on the feature information acquired by analyzing the captured image in S15 corresponds to “identification means”. The bone conduction earphone of the headset 35 corresponds to a “sound generator”. The CPU 41 that outputs the sound based on the instruction information in S37 corresponds to an “output unit”. The CPU 41 detects completion of the work in S29, and returns to S11 when there is a next target part in S31. Thus, the CPU 41 executes S25 and displays the procedure information of the next target part in S27. It corresponds to “switching means”.

1 Head Mount Display 8 Half Mirror 35 Headset 36 Image Display Unit 39 Camera

Claims (8)

A display device configured to be mounted on the operator's head and capable of displaying an image;
A photographing device capable of photographing a work object on which the worker performs work;
The operator acquires the procedure information for displaying the procedure of the work to be performed on the work object on the display device as an image by facing the work object side from a predetermined direction with respect to the work object. Acquisition means;
A second acquisition means for acquiring a predetermined image, which is an image of a predetermined part of the work object, from the captured image captured by the imaging device as specific information for specifying an orientation;
Determining means for determining a current orientation of the worker with respect to the work object based on the specific information;
A first display means for displaying an image based on the procedure information on the display device when the current orientation of the worker is the predetermined orientation targeted in the work procedure indicated by the procedure information;
An instruction determined according to the relationship between the current direction of the worker and the predetermined direction when the current direction of the worker is not the predetermined direction targeted in the work procedure indicated by the procedure information Second display means for displaying information on the display device;
A work assistance system characterized by comprising: The display device reflects at least a part of image light emitted from an image forming element that forms an image and enters the eye of the worker, and transmits at least a part of light from an external real image. A display unit for visually recognizing the image superimposed on a real image in the field of view of the worker,
The said 2nd display means displays the said instruction information on the display position in the said display part preset according to the said operator's present azimuth | direction and the said predetermined | prescribed azimuth | direction. Work assistance system as described. The imaging device is configured to be attachable to the operator's head, and a shooting direction and an angle of view are set in accordance with an external field of view visible through the display unit, out of the worker's field of view. Has been
Further comprising a specifying means for specifying a position where a real image of the work object is displayed in the display unit from the captured image captured by the imaging device;
The second display means uses the position of the work object in the display unit specified by the specifying means as a reference position, and in advance with respect to the reference position according to the current direction of the worker and the predetermined direction. The work assistance system according to claim 2, wherein a display position set relatively is applied to the display unit to display the instruction information. Based on the specific information acquired by the second acquisition means, further comprising identification means for identifying the work object;
4. The work assistance system according to claim 1, wherein the first acquisition unit acquires the procedure information corresponding to the work object identified by the identification unit. 5. The instruction information is information including at least one of an image and characters using arrows,
The route of movement to the predetermined direction instructed by the operator by the instruction information is a route in which a difference between the current direction of the worker and the predetermined direction is minimum. Item 5. The work assistance system according to any one of Items 1 to 4.   When the instruction information is information indicating with an image using an arrow, the second display means displays an arrow having a length corresponding to a difference between the current direction of the worker and the predetermined direction. The work assistance system according to claim 5, wherein the work assistance system is displayed on an apparatus.   The work assistance system according to claim 1, further comprising a switching unit that switches display contents of an image based on the procedure information by the first display unit. An image is displayed on a display device configured to be attachable to the worker's head based on a photographed image obtained by photographing the work object with a photographing device, and the worker performs work on the work object. A program for functioning as a work assistance device for assisting the worker in the case,
On the computer,
Procedure information for the operator to face the work object side from a predetermined orientation with respect to the work object and to display the procedure of the work to be performed on the work object on the display device as an image is acquired. An acquisition step;
A second acquisition step in which a predetermined image, which is an image of a predetermined part of the work object, is acquired as specific information for specifying an orientation from a captured image captured by the imaging device;
A determination step in which a current orientation of the worker with respect to the work object is determined based on the captured image and the specific information;
A first display step in which an image based on the procedure information is displayed on the display device when the current orientation of the worker is the predetermined orientation targeted in the work procedure indicated by the procedure information;
An instruction determined according to the relationship between the current direction of the worker and the predetermined direction when the current direction of the worker is not the predetermined direction targeted in the work procedure indicated by the procedure information A second display step in which information is displayed on the display device;
A program that executes

Images