JP6631203B2 - Virtual fence display system - Google Patents

Description

  The present invention relates to a virtual fence display system for displaying, on a display unit worn by an operator on a head, 3D model image data obtained by modeling a safety fence arranged around a robot body in three dimensions.

  For example, Patent Literature 1 discloses a technology of displaying a virtual reality image and various information using a head-mounted display configured to project an image on a display unit in a state where the worker is mounted on the head. Proposed. It is assumed that this technique is applied to ensure the safety of a worker who may enter an area where the robot works.

  For example, as one method, it is conceivable to display an image of a virtual safety fence around the robot when the operator views the robot through the head-mounted display. Hereinafter, this “virtual safety fence” is referred to as “virtual fence”. By introducing such a system, it is possible to ensure the safety of workers, and it is not necessary to actually arrange a safety fence, so it is expected that the work efficiency of workers in an environment where robots coexist will be improved. it can.

JP 2014-95903 A

  However, on the other hand, if the virtual fence is always displayed on the head-mounted display when the worker sees the robot in the field of view, the work efficiency may be reduced due to distraction of the worker. is there.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a virtual fence display capable of optimizing a display mode of a virtual fence on a head-mounted display so as not to affect the work efficiency of an operator as much as possible. It is to provide a system.

  According to the virtual fence display system of the first aspect, the image processing unit is a 3D model image data obtained by modeling the safety fence arranged around the robot main body in three dimensions around the position of the robot main body, that is, a virtual fence. Holds 3D model image data of the fence. The image processing unit is capable of processing the 3D model image data of the virtual fence in such a form as to be divided into a plurality of rectangular parallelepipeds, and displaying each rectangular parallelepiped as an image having a predetermined transmittance. It should be noted that the term “cuboid” also includes “cube”.

When the image processing unit acquires the information on the position and direction of the worker wearing the head-mounted display, the 3D model image data of the virtual fence to be displayed on the display unit moves into the field of view of the worker. Process to form. At that time, depending on the relationship between the position and direction between the virtual fence and the worker, among a plurality of rectangular parallelepipeds, the one closer to the worker has a lower transmittance, and the one farther away has a lower transmittance. It is processed to be higher and wirelessly transmitted to the head-mounted display side. Then, the head-mounted display projects the received 3D model image data of the virtual fence on the display unit.

  With this configuration, the image of the entire virtual fence is not always displayed on the head-mounted display, and the rectangular parallelepiped portion relatively close to the worker has low transmittance, and the It is displayed in a state where the worker is aware of the existence. On the other hand, the rectangular parallelepiped portion at a position relatively far from the worker has a high transmittance, and is displayed in a state where the worker is not so conscious of the existence of the virtual fence.

  Therefore, even if a safety fence is not arranged around the actual robot body, it is possible to ensure the safety of the worker by displaying the virtual fence, and in an area where the robot and the worker coexist and work. The working efficiency of the worker can be improved. In addition, the portion of the virtual fence at a relatively distant position where the safety of the worker is sufficiently ensured has a high transmittance, so that it is possible to prevent the worker from being distracted. In addition, the worker can easily visually recognize the actual scene around the robot main body and its surroundings, so that improvement in working efficiency can be expected from that point.

  According to the virtual fence display system of the second aspect, the position / direction information acquisition unit is configured to detect the position / direction information acquisition unit from the image of the robot body included in the image data captured by the imager arranged on the head-mounted display and the image data captured by the imager. , And an image processing unit that acquires information on the position and direction of the worker.

  It is easy to place a relatively small imager on the head-mounted display, and from the image data captured by the imager, the robot body can be captured in any direction and size in the field of view of the worker. Can be grasped. Accordingly, since the image processing unit can obtain the information on the position and direction of the worker by processing the image data, it is not necessary to use a separate sensor for obtaining the information.

  According to the virtual fence display system of the third aspect, the image processing unit changes the transmittance of the 3D model image data in stages. Here, the “predetermined stage” is a stage at least larger than the minimum resolution when the image processing unit changes the transmittance, and is a stage at which the operator can recognize the change in the transmittance. With this configuration, the amount of calculation in the image processing unit can be reduced, and the operator can more clearly recognize the boundary where the transmittance of the rectangular parallelepiped changes.

1 is a functional block diagram schematically illustrating a configuration of a virtual fence display system according to an embodiment. Perspective view showing the configuration of a smart glass FIG. 4 is a diagram illustrating an example of a field of view seen by a worker wearing smart glasses through a display unit. Flow chart mainly showing the processing contents by the robot safety controller The figure which shows the setting example of the transmittance | permeability of each rectangular parallelepiped in plane. The figure which shows the setting example of the transmittance | permeability of each rectangular parallelepiped three-dimensionally

  Hereinafter, an embodiment will be described with reference to the drawings. As shown in FIG. 1, the virtual fence display system 1 of the present embodiment includes, for example, a robot arm 2 for assembly, a robot safety controller 3, a smart glass 4, and a camera 5. The robot arm 2, which is a robot main body, is configured as, for example, a 6-axis vertical articulated robot. Although a detailed description of the general configuration is omitted, the robot arm 2 has six-axis arms each driven by a servomotor, and is housed at the tip of the sixth-axis arm, for example, in a pallet. A hand or the like for gripping the work is provided.

  The robot arm 2 is connected via a cable 6 to a robot safety controller 3 also having a function as a robot controller, and the servo motors of the respective axes are controlled by the robot safety controller 3. The robot safety controller 3 corresponding to the image processing unit acquires information on the three-dimensional position coordinates (x, y, z) of the robot arm 2 that is held in advance. Further, the robot safety controller 3 stores 3D model image data, which is data obtained by three-dimensionally modeling the form of the virtual fence VF, which is a safety fence virtually arranged around the robot arm 2, in an internal memory. And hold.

  As shown in FIG. 1, a smart glass 4 which is a head-mounted display is worn by an operator 7 like eyeglasses on a head, and is shown on a transparent display unit 4D corresponding to a lens portion of the eyeglasses. This is a so-called transmissive display capable of projecting an image via a projection unit that does not. As shown in FIG. 2, a position / direction information output unit and a camera 5 serving as an image pickup device are arranged on one side of the frame of the smart glass 4. The camera 5 captures an image in the direction in which the front of the head of the worker 7 is facing, with the worker 7 wearing the smart glass 4 on the head. For example, the camera 5 is a CCD (Charge Coupled Device) or a CMOS image. It is composed of sensors and the like.

  The smart glass 4 has a built-in wireless communication unit 8 shown by an antenna symbol in FIGS. 1 and 2 in a frame, and can communicate with the wireless communication unit 9 of the robot safety controller 3 also shown by an antenna symbol in FIG. Has become. The wireless communication unit 8 transmits data of an image captured by the camera 5 to the robot safety controller 3, and the wireless communication unit 9 transmits the 3D model image data held by the robot safety controller 3 to the smart glass 4. I do. The communication units 8 and 9 correspond to the first and second communication units, respectively.

  Here, the 3D model image data of the virtual fence VF is modeled in a state where the entire virtual fence VF is divided by a plurality of rectangular parallelepipeds Cd, as shown in FIGS. Then, image processing can be performed so that each of the rectangular parallelepipeds Cd is individually displayed at a different transmittance.

  FIG. 3 shows a state of the field of view seen by the worker 7 wearing the smart glass 4 through the display unit 4D. For the robot arm 2N near the worker 7, a 3D model image of the virtual fence VF is displayed. However, the whole is not displayed, and the rectangular parallelepiped Cd of the portion of the virtual fence VF that is relatively close to the worker 7 is drawn with a relatively low transmittance, and is given to the worker 7. The rectangular parallelepiped Cd at a relatively distant position is drawn with a relatively high transmittance. The robot arm 2F located far from the worker 7 does not affect the safety when the worker 7 works, so that the 3D model image of the virtual fence VF is not displayed. That is, the transmittance of all the rectangular parallelepipeds Cd is set to 100%. The above “distant” is, for example, a distance farther than a region where the robot arm 2 works.

  Next, the operation of the present embodiment for displaying an image on the smart glass 4 as described above will be described with reference to FIGS. FIG. 4 is a flowchart mainly showing the processing contents of the robot safety controller 3. First, the robot safety controller 3 acquires the coordinate position of the worker 7 from the smart glass 4 (S1). Specifically, the smart glass 4 transmits data of an image captured by the camera 5 to the robot safety controller 3. If the image data includes the robot arm 2, the robot safety controller 3 specifies the relative coordinate position of the worker 7 from the three-dimensional form of the robot arm 2.

  Subsequently, when the coordinate position of the robot arm 2 is acquired (S2), the position of the robot arm 2 is matched with the position of the worker 7 on world coordinates, and the positional relationship between them is specified (S3). Then, the distance between the robot arm 2 and the worker 7 and the direction in which the robot arm 2 is seen from the viewpoint of the worker 7 are calculated (S4).

  In the following step S5, it is determined whether or not the calculated distance and direction are within a range in which the virtual fence VF needs to be displayed on the smart glass 4. If it is not within the range (NO), the process ends. If it is within the range (YES), the drawing area of the virtual fence VF is divided into a plurality of blocks, that is, a rectangular parallelepiped Cd (S6).

  The subsequent loop of steps S7 to S10 is a process for each rectangular parallelepiped Cd. When the distance from the position of the worker 7 to each cuboid Cd is calculated (S8), the transmittance of each cuboid Cd is determined according to the calculated distance (S9). After processing all the rectangular parallelepipeds Cd, the robot safety controller 3 transmits the 3D model image data of the virtual fence VF and the drawing command thereof to the smart glass 4 (S11). The smart glass 4 that receives them draws and displays a 3D image of the virtual fence VF on the display unit 4D (S12).

  FIG. 5 is a plan view showing a specific example of setting the transmittance by the above processing. It is assumed that the worker 7 is located at the center of the virtual fence VF in front of the robot arm 2 and is within a predetermined distance range for displaying the virtual fence VF. In a plan view, the virtual fence VF is divided into 12 parts of 3 × 4 rectangular parallelepipeds Cd. At this time, if the transmittance of the rectangular parallelepipeds Cd (1, 2) and (1, 3) in the center of the first row closest to the worker 7 is, for example, 25%, the rectangular parallelepipeds Cd (1, The transmittances of 1) and (1, 4) are, for example, 50%.

  Assuming that the transmittance of the rectangular parallelepipeds Cd (2,2) and (2,3) at the center of the next second row is, for example, 50%, the rectangular parallelepipeds Cd (2,1) and (2,4) located outside the rectangular parallelepipeds Cd (2,2) and (2,4). ) Is, for example, 75%. Then, the third row of the rectangular parallelepipeds Cd (3, 1) to (3, 4) behind the robot arm is set to a transmittance of 100% and is not displayed.

Although FIG. 6 shows the virtual fence VF in three dimensions, in this case, the virtual fence VF is divided into nine by 3 × 3 in a plane and divided into two in the height direction to be three-dimensionally divided into eighteen. . As shown in FIG. 6C, when the worker 7 is located at the center of the virtual fence VF, the rectangular parallelepipeds Cd (1,1,1) to (1,3,1) below the first row are provided. Has a transmittance of 25%. The following is a tabulated summary.
Transmissivity Upper third row; (3,1,2) to (3,3,2) 100%
2nd row above; (2,1,2) to (2,3,2) 75%
First row above; (1,1,2) to (1,3,2) 50%
Lower third row; (3,1,1) to (3,3,1) 100%
2nd row lower; (2,1,1)-(2,3,1) 50%
First row lower; (1,1,1) to (1,3,1) 25%

As shown in FIG. 6B, when the worker 7 is located on the left side of the virtual fence VF, the operation is as follows.
Transmissivity Upper third row; (3,1,2) to (3,3,2) 100%, 100%, 100%
(2,1,2) to (2,3,2) 75%, 100%, 100%
Upper first row; (1,1,2) to (1,3,2) 50%, 75%, 100%
(3,1,1) to (3,3,1) 75%, 100%, 100%
(2,1,1)-(2,3,1) 50%, 75%, 100%
1st row below; (1,1,1)-(1,3,1) 25%, 50%, 75%

As shown in FIG. 6D, when the worker 7 is located on the right side of the virtual fence VF, the operation is as follows.
Transmissivity Upper third row; (3,1,2) to (3,3,2) 100%, 100%, 100%
(2,1,2) to (2,3,2) 100%, 100%, 75%
Upper first row; (1,1,2) to (1,3,2) 100%, 75%, 50%
Lower third row; (3,1,1) to (3,3,1) 100%, 100%, 75%
(2,1,1)-(2,3,1) 100%, 75%, 50%
1st row lower; (1,1,1)-(1,3,1) 75%, 50%, 25%

  By setting the transmittance of each of the rectangular parallelepipeds Cd in this manner, the portion of the virtual fence VF relatively close to the worker 7 is easily recognized in the field of view of the worker 7, and the worker 7 can recognize the virtual fence VF. Makes people more aware of ensuring their own safety during work. On the other hand, the portion of the virtual fence VF that is relatively far from the worker 7 is hardly recognized in the field of view of the worker 7. Therefore, the worker 7 can easily visually recognize the robot arm 2 and the actual scene around it without being hindered by the virtual fence VF.

  As described above, according to the present embodiment, the robot safety controller 3 holds the 3D model image data of the virtual fence VF that is virtually arranged around the position of the robot arm 2 and its 3D model. Image data is divided into a plurality of rectangular parallelepipeds Cd, and each rectangular parallelepiped Cd can be displayed as an image having a predetermined transmittance.

  Then, when the robot safety controller 3 acquires the information on the position and direction of the worker 7 wearing the smart glass 4, the 3D model image data of the virtual fence to be displayed on the display unit 4D moves to the view of the worker 7. It is processed so as to be in a form to fit. At that time, depending on the relationship between the position and direction between the virtual fence VF and the worker 7, among the plurality of rectangular parallelepipeds Cd, the one closer to the worker 7 has a lower transmittance and the one farther away. The higher the transmittance, the higher the transmittance and the transmission to the smart glass 4 side. Then, the smart glass 4 projects and displays the received 3D model image data of the virtual fence on the display unit 4D.

  Accordingly, the safety of the worker 7 can be ensured by displaying the virtual fence VF without disposing the safety fence around the actual robot arm 2 and the robot and the worker 7 coexist. The work efficiency of the worker 7 in the work area can be improved. Further, the portion of the virtual fence VF at a relatively distant position where the safety of the worker 7 is sufficiently ensured has a high transmittance, so that the worker 7 can be prevented from being distracted. In addition, the worker 7 can easily visually recognize the robot arm 2 and the actual scene around the robot arm 2, and from that point, an improvement in working efficiency can be expected.

  Then, the robot safety controller 3 acquires information on the position and direction of the worker 7 from the image of the robot arm 2 included in the image data captured by the camera 5 disposed on the smart glass 4. That is, it is easy to dispose the small camera 5 on the smart glass 4, and from the image data captured by the camera 5, the robot arm 2 can be captured in any direction and size in the field of view of the worker 7. Can be grasped. Therefore, by processing the image data by the robot safety controller 3, information on the position and direction of the worker 7 can be acquired, and it is not necessary to use a separate sensor for acquiring the information.

  Further, the robot safety controller 3 changes the transmittance of the 3D model image data in stages. Here, the “predetermined stage” is a stage that is at least larger than the minimum resolution when the robot safety controller 3 changes the transmittance and that the operator 7 can recognize the change in the transmittance. With this configuration, the amount of calculation in the robot safety controller 3 can be reduced, and the operator 7 can more clearly recognize the boundary where the transmittance of the rectangular parallelepiped Cd changes.

The present invention is not limited only to the embodiments described above or illustrated in the drawings, and the following modifications or extensions are possible.
The specific numerical value of the transmittance may be appropriately changed according to individual designs.
The rectangular parallelepiped Cd may be a cube.
The robot main body is not limited to the robot arm 2 and may be, for example, a horizontal 4-axis robot arm, a self-propelled robot, or a humanoid robot.

A laser sensor or an infrared sensor, or each sensor such as a GPS, a gyro, and an acceleration may be used for the position / direction information acquisition unit. Then, these sensor signals may be directly input to the robot safety controller 3.
The head-mounted display does not necessarily need to be the smart glass 4, but may be any as long as the image can be projected on a display unit worn by the operator on the head.

  In the drawing, 1 is a virtual fence display system, 2 is a robot arm, 3 is a robot safety controller, 4 is a smart glass, 4D is a display unit, 5 is a camera, 7 is an operator, and 8 and 9 are wireless communication units.

Claims (3)

A head-mounted display having a display unit worn by the operator on the head, and configured to project an image on the display unit ;
An image processing unit that holds 3D model image data obtained by modeling a safety fence three-dimensionally arranged around the robot main body around the position of the robot main body;
A position and direction information acquisition unit that acquires information on the position of the worker and the direction in which the front of the head of the worker is facing,
The image processing unit is capable of processing so that the 3D model image data of the safety fence is divided into a plurality of rectangular parallelepipeds, and each rectangular parallelepiped is displayed as an image having a predetermined transmittance.
When information on the position and direction of the worker is obtained via the position and direction information obtaining unit, the 3D model image data of the safety fence to be displayed on the head-mounted display enters the field of view of the worker. While processing to become
According to the relationship between the position and the direction of the safety fence and the worker, of the plurality of rectangular parallelepipeds, the one closer to the worker has a lower transmittance, and the cuboid is farther away from the worker. engineered that higher transmittance increases what is on the side, wirelessly transmitted before Symbol head mounted display side,
The virtual fence display system, wherein the head-mounted display receives the processed 3D model image data of the safety fence and projects the 3D model image data on the display unit . The position and direction information acquisition unit, the head mounted display, an imager arranged to capture an image of the worker's head front direction,
The image processing unit configured to acquire information on a position and a direction of the worker from an image of the robot body included in the image data when the image data acquired by the imager is acquired. The virtual fence display system according to 1.   The virtual fence display system according to claim 1, wherein the image processing unit changes the transmittance of the 3D model image data in stages.

Images