JP2017100205A - Virtual fence display system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a head-mounted display with a virtual fence display system.SOLUTION: A robot safety controller 3 holds 3D model image data of a virtual fence VF virtually arranged around with the position of a robot arm 2 as a center, and can display the image data in modes divided by a plurality of rectangular parallelepipeds and each rectangular parallelepiped by each image having a prescribed transmittance. When acquiring information on the position and the direction of a worker 7 wearing smart glasses 4, the controller 3 processes the 3D model image data of the virtual fence to be displayed in a display part such that the closer to the worker 7 a rectangular parallelepiped is located among the plurality of rectangular parallelepipeds, the lower transmittance is, and the farther a rectangular parallelepiped is located, the higher transmittance is in accordance with a relation in positions and directions between the virtual fence VF and the worker 7, and transmits the image data to the smart glasses 4 side. The smart glasses 4 project the received 3D model image data to be displayed in the display part.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a virtual fence display system for displaying 3D model image data obtained by modeling a safety fence arranged around a robot body in a three-dimensional manner on a display unit worn by an operator on the head.

  A technique for displaying a virtual reality image and various information using, for example, a head-mounted display configured to be able to project an image on a display unit worn by the operator on the head is disclosed in, for example, Patent Document 1 Proposed. It is assumed that this technology is applied to ensure the safety of an operator who may enter an area where the robot works.

  For example, as one method, when an operator views a robot via a head-mounted display, it is conceivable to display a virtual safety fence image around the robot. Hereinafter, this “virtual safety fence” is referred to as a “virtual fence”. By introducing such a system, it is possible to ensure the safety of the worker, and in fact, it is not necessary to install a safety fence, so it is expected that the work efficiency of the worker in an environment coexisting with the robot will be improved. it can.

JP 2014-95903 A

  However, on the other hand, when the worker catches the robot in the field of view, if the virtual fence is always displayed on the head-mounted display, the operator's distraction may cause a reduction in work efficiency. is there.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a virtual fence display that can optimize the display mode of the virtual fence on the head-mounted display so as not to affect the work efficiency of the operator as much as possible. To provide a system.

  According to the virtual fence display system of claim 1, the image processing unit 3D model image data obtained by modeling a safety fence arranged around the robot body around the position of the robot body in three dimensions, that is, virtual Holds 3D model image data of the fence. The image processing unit can process the 3D model image data of the virtual fence in a form obtained by dividing the virtual fence into a plurality of rectangular parallelepipeds and display each rectangular parallelepiped as an image having a predetermined transmittance. Here, the “cuboid” includes “cube”.

  When the image processing unit acquires information on the position and direction of the worker wearing the head-mounted display, the virtual fence 3D model image data to be displayed on the display unit moves into the worker's field of view. Process to form. At that time, according to the relationship between the position and direction of the virtual fence and the worker, among the plurality of rectangular parallelepipeds, the one closer to the worker has a lower transmittance, and the one closer to the worker has a lower transmittance. Process it to be higher and send it to the head-mounted display. Then, the head-mounted display projects the received 3D model image data of the virtual fence on the display unit.

  With this configuration, the image of the entire virtual fence is not always displayed on the head-mounted display, and the rectangular parallelepiped portion located relatively close to the worker has low transmittance, and the virtual fence Displayed in a state in which the presence is made more conscious by the worker. On the other hand, the rectangular parallelepiped portion located at a position relatively far from the worker has a high transmittance, and is displayed in a state in which the worker is not conscious of the existence of the virtual fence.

  Therefore, it is possible to ensure the safety of the worker by displaying the virtual fence without arranging a safety fence around the actual robot body, and in the area where the robot and the worker coexist. The work efficiency of the worker can be improved. Further, since the transmittance of the portion of the virtual fence at a relatively far position where the safety of the worker is sufficiently secured is high, it is possible to prevent the worker from being distracted. In addition, since it becomes easier for the worker to visually recognize the actual scene of the robot body and its surroundings, improvement in work efficiency can also be expected from this point.

  According to the virtual fence display system of claim 2, the position / direction information acquisition unit is configured based on the image of the robot body included in the image data captured by the image sensor and the image sensor that is arranged on the head-mounted display. And an image processing unit that acquires information on the position and direction of the worker.

  It is easy to place a relatively small imager on a head-mounted display. From the image data captured by the imager, the orientation and size of the robot body can be captured in the operator's field of view. You can see if you are. Therefore, since the image processing unit can acquire the information on the position and direction of the worker by processing the image data, it is not necessary to separately use a sensor for acquiring the information.

  According to the virtual fence display system of the third aspect, the image processing unit changes the transmittance of the 3D model image data in a stepwise manner. The “present stage” here is a stage larger than at least the minimum resolution when the image processing unit changes the transmittance, and is a stage where the operator can recognize the change in the transmittance. With this configuration, it is possible to reduce the amount of calculation in the image processing unit and to make the operator more clearly recognize the boundary where the transmittance of the rectangular parallelepiped changes.

1 is a functional block diagram schematically showing a configuration of a virtual fence display system according to an embodiment Perspective view showing configuration of smart glass The figure which shows an example of the field of view which the operator who wore the smart glass looked through the display part Flowchart showing the details of processing by the robot safety controller The figure which shows the example of setting the transmittance of each rectangular parallelepiped The figure which shows the setting example of the transmittance of each rectangular parallelepiped in three dimensions

  Hereinafter, an embodiment will be described with reference to the drawings. As shown in FIG. 1, the virtual fence display system 1 of the present embodiment includes, for example, a robot arm 2 for assembly, a robot safety controller 3, a smart glass 4 and a camera 5. The robot arm 2 which is a robot body is configured as, for example, a 6-axis vertical articulated robot. Although a detailed description of a general configuration is omitted, the robot arm 2 has 6-axis arms each driven by a servo motor, and is housed in, for example, a pallet at the tip of the sixth-axis arm. A hand for gripping a workpiece is provided.

  The robot arm 2 is connected to a robot safety controller 3 that also has a function as a robot controller via a cable 6, and the servo motor of each axis is controlled by the robot safety controller 3. The robot safety controller 3 corresponding to the image processing unit acquires information on the three-dimensional position coordinates (x, y, z) of the robot arm 2 held in advance. Further, the robot safety controller 3 stores 3D model image data, which is data obtained by three-dimensionally modeling the form of the virtual fence VF, which is a safety fence virtually arranged around the robot arm 2, in an internal memory. And hold.

  As shown in FIG. 1, a smart glass 4 that is a head-mounted display is worn by an operator 7 on the head like glasses, and is displayed on a transparent display unit 4D corresponding to a lens portion of the glasses. This is a so-called transmissive display capable of projecting an image via a non-projecting unit. On one side of the frame of the smart glass 4, as shown in FIG. 2, a position / direction information output unit and a camera 5 that is an image pickup device are arranged. The camera 5 captures an image in a direction in which the worker 7 wears the smart glass 4 on the head and the front surface of the worker 7 is facing, for example, a CCD (Charge Coupled Device) or a CMOS image. It consists of sensors.

  The smart glass 4 incorporates a wireless communication unit 8 indicated by an antenna symbol in FIGS. 1 and 2 in the frame, and can communicate with the wireless communication unit 9 of the robot safety controller 3 indicated by the antenna symbol in FIG. It has become. The wireless communication unit 8 transmits image data captured by the camera 5 to the robot safety controller 3, and the wireless communication unit 9 transmits the 3D model image data held by the robot safety controller 3 to the smart glass 4. To do. The communication units 8 and 9 correspond to first and second communication units, respectively.

  Here, the 3D model image data of the virtual fence VF is modeled in a state where the entire virtual fence VF is divided by a plurality of rectangular parallelepipeds Cd as shown in FIGS. Then, each rectangular parallelepiped Cd can be image-processed so as to be displayed with different transmittances.

  FIG. 3 shows the state of the field of view as seen through the display unit 4D by the operator 7 wearing the smart glass 4. For the robot arm 2N in the vicinity of the worker 7, a 3D model image of the virtual fence VF is displayed. However, the whole is not displayed, and the rectangular parallelepiped Cd at a position relatively close to the worker 7 in the virtual fence VF is drawn with a setting with a relatively low transmittance. The rectangular parallelepiped Cd at a relatively far position is drawn with a setting with a relatively high transmittance. Since the robot arm 2F located far from the worker 7 does not affect the safety when the worker 7 works, the 3D model image of the virtual fence VF is not displayed. That is, the transmittance of all cuboids Cd is set to 100%. The above “far” is, for example, a distance away from a region where the robot arm 2 is working.

  Next, the operation of this embodiment for displaying an image on the smart glass 4 as described above will be described with reference to FIGS. FIG. 4 is a flowchart showing mainly the contents of processing by the robot safety controller 3. The robot safety controller 3 first acquires the coordinate position of the worker 7 from the smart glass 4 (S1). Specifically, the smart glass 4 transmits image data captured by the camera 5 to the robot safety controller 3. When the robot arm 2 is included in the image data, the robot safety controller 3 specifies the relative coordinate position of the operator 7 from the three-dimensional form of the robot arm 2.

  Subsequently, when the coordinate position of the robot arm 2 is acquired (S2), the position of the robot arm 2 and the position of the operator 7 are matched on the world coordinates, and the positional relationship between the two is specified (S3). Then, the distance between the robot arm 2 and the worker 7 and the direction in which the robot arm 2 can be seen from the viewpoint of the worker 7 are calculated (S4).

  In a subsequent step S5, it is determined whether or not the calculated distance and direction are within a range where the virtual fence VF needs to be displayed on the smart glass 4. If it is not within the range (NO), the process is terminated. If it is within the range (YES), the drawing area of the virtual fence VF is divided into a plurality of blocks, that is, a rectangular parallelepiped Cd (S6).

  The subsequent loop of steps S7 to S10 is processing for each rectangular parallelepiped Cd. When the distance from the position of the worker 7 to each rectangular parallelepiped Cd is calculated (S8), the transmittance of each rectangular parallelepiped Cd is determined according to the calculated distance (S9). When all the rectangular parallelepipeds Cd are processed, the robot safety controller 3 transmits the 3D model image data of the virtual fence VF and its drawing command to the smart glass 4 (S11). The smart glass 4 that has received them draws and displays a 3D image of the virtual fence VF on the display unit 4D (S12).

  FIG. 5 is a plan view showing a specific example of transmittance setting by the above processing. It is assumed that the worker 7 is located in the center of the virtual fence VF in front of the robot arm 2 and is within a predetermined distance range for displaying the virtual fence VF. In plan view, the virtual fence VF is divided into 12 pieces of 3 × 4 rectangular parallelepiped Cd. At this time, assuming that the transmittance of the rectangular parallelepipeds Cd (1,2) and (1,3) in the center of the first row closest to the worker 7 is 25%, for example, the rectangular parallelepiped Cd (1, The transmittance of 1) and (1, 4) is set to 50%, for example.

  If the transmittance of the rectangular parallelepipeds Cd (2,2) and (2,3) in the center of the next second row is 50%, for example, the rectangular parallelepipeds Cd (2,1) and (2,4) outside them ) Is, for example, 75%. Then, the third row of rectangular parallelepipeds Cd (3, 1) to (3, 4) behind the robot arm are not displayed with a transmittance of 100%.

In addition, in FIG. 6, the virtual fence VF is divided into 9 parts by 3 × 3 on a plane and divided into 18 parts by dividing into two in the height direction in this case. . As shown in FIG. 6C, when the worker 7 is located at the center of the virtual fence VF, the rectangular parallelepipeds Cd (1, 1, 1) to (1, 3, 1) located below the first row. Has a transmittance of 25%. The table below summarizes.
Transmittance Upper third row; (3, 1, 2) to (3, 3, 2) 100%
Upper second row; (2,1,2) to (2,3,2) 75%
Upper first row; (1,1,2) to (1,3,2) 50%
Lower third row; (3, 1, 1) to (3, 3, 1) 100%
Second row down; (2,1,1) to (2,3,1) 50%
1st row down; (1,1,1) to (1,3,1) 25%

As shown in FIG. 6B, when the worker 7 is located on the left side of the virtual fence VF, the following occurs.
Transmittance Upper third row; (3, 1, 2) to (3, 3, 2) 100%, 100%, 100%
2nd row upper; (2,1,2) to (2,3,2) 75%, 100%, 100%
Upper first row; (1,1,2) to (1,3,2) 50%, 75%, 100%
Lower third row; (3, 1, 1) to (3, 3, 1) 75%, 100%, 100%
Second row down; (2,1,1) to (2,3,1) 50%, 75%, 100%
Lower row 1; (1,1,1) to (1,3,1) 25%, 50%, 75%

As shown in FIG. 6D, when the worker 7 is located on the right side of the virtual fence VF, the following occurs.
Transmittance Upper third row; (3, 1, 2) to (3, 3, 2) 100%, 100%, 100%
Second row above; (2,1,2) to (2,3,2) 100%, 100%, 75%
Upper first row; (1,1,2) to (1,3,2) 100%, 75%, 50%
Lower third row; (3, 1, 1) to (3, 3, 1) 100%, 100%, 75%
Second row down; (2,1,1) to (2,3,1) 100%, 75%, 50%
1st row down; (1,1,1) to (1,3,1) 75%, 50%, 25%

  By setting the transmittance of each rectangular parallelepiped Cd in this manner, the portion of the virtual fence VF that is relatively close to the worker 7 can be easily recognized in the field of view of the worker 7, and the worker 7 can see the virtual fence VF. Because of the presence of, it becomes more conscious of ensuring their own safety during work. On the other hand, the portion of the virtual fence VF that is relatively far from the worker 7 is difficult to be recognized in the field of view of the worker 7. Therefore, the operator 7 can easily visually recognize the robot arm 2 and the surrounding scene without being interrupted by the virtual fence VF.

  As described above, according to the present embodiment, the robot safety controller 3 holds the 3D model image data of the virtual fence VF that is virtually arranged around the position of the robot arm 2 and that 3D model. The image data is divided into a plurality of rectangular parallelepipeds Cd, and each rectangular parallelepiped Cd can be displayed as an image having a predetermined transmittance.

  When the robot safety controller 3 acquires information on the position and direction of the worker 7 wearing the smart glass 4, the 3D model image data of the virtual fence displayed on the display unit 4D is transferred to the field of view of the worker 7. To be in the form of At that time, according to the relationship between the position and direction of the virtual fence VF and the worker 7, among the plurality of rectangular parallelepiped Cd, the closer to the worker 7, the lower the transmittance, and the farther side It is processed so that the transmittance becomes higher and transmitted to the smart glass 4 side. Then, the smart glass 4 projects and displays the received 3D model image data of the virtual fence on the display unit 4D.

  Therefore, the safety of the worker 7 can be ensured by displaying the virtual fence VF without arranging a safety fence around the actual robot arm 2, and the robot and the worker 7 coexist. The work efficiency of the worker 7 in the work area can be improved. Moreover, since the transmittance | permeability becomes high in the part of the virtual fence VF in the relatively distant position where the safety | security of the operator 7 is fully ensured, it can prevent the operator 7 being distracted. In addition, since the operator 7 can easily see the actual scene of the robot arm 2 and its surroundings, improvement in work efficiency can also be expected from this point.

  Then, the robot safety controller 3 acquires information on the position and direction of the operator 7 from the image of the robot arm 2 included in the image data captured by the camera 5 disposed on the smart glass 4. That is, it is easy to arrange the small camera 5 on the smart glass 4, and the direction and size of the robot arm 2 can be captured from the image data captured by the camera 5 in the field of view of the operator 7. You can see if you are. Therefore, when the robot safety controller 3 processes the image data, information on the position and direction of the operator 7 can be acquired, and it is not necessary to separately use a sensor for acquiring the information.

  Further, the robot safety controller 3 changes the transmittance of the 3D model image data in a stepped manner. The “present stage” here is a stage that is at least larger than the minimum resolution when the robot safety controller 3 changes the transmittance and that allows the operator 7 to recognize the change in the transmittance. With this configuration, the amount of calculation in the robot safety controller 3 can be reduced, and the operator 7 can more clearly recognize the boundary where the transmittance of the rectangular parallelepiped Cd changes.

The present invention is not limited to the embodiments described above or shown in the drawings, and the following modifications or expansions are possible.
What is necessary is just to change suitably the specific numerical value of the transmittance | permeability according to an individual design.
The rectangular parallelepiped Cd may be a cube.
The robot body is not limited to the robot arm 2, but may be a robot arm having a horizontal 4-axis configuration, a self-propelled robot, or a humanoid robot.

For the position / direction information acquisition unit, a laser sensor or an infrared sensor, or each sensor such as GPS, gyroscope, acceleration, etc. These sensor signals may be directly input to the robot safety controller 3.
The head-mounted display is not necessarily the smart glass 4 and may be any display that can project an image on a display unit worn by the operator on the head.

  In the drawings, 1 is a virtual fence display system, 2 is a robot arm, 3 is a robot safety controller, 4 is a smart glass, 4D is a display unit, 5 is a camera, 7 is an operator, and 8 and 9 are wireless communication units.

Claims (3)

A head-mounted display configured to be able to project an image on a display unit worn by the operator on the head; and
An image processing unit that holds 3D model image data obtained by modeling a safety fence arranged around the robot body around the position of the robot body in three dimensions;
A position and direction information acquisition unit that acquires information on the position of the worker and the direction in which the front of the head of the worker is facing;
The image processing unit can process the 3D model image data of the safety fence in a form divided by a plurality of rectangular parallelepipeds and display each rectangular parallelepiped as an image having a predetermined transmittance.
When the information on the position and direction of the worker is acquired via the position / direction information acquisition unit, the 3D model image data of the safety fence displayed on the head-mounted display moves to the field of view of the worker While processing to become
Depending on the relationship between the position and direction of the safety fence and the worker, among the plurality of rectangular parallelepipeds, the closer to the worker, the lower the transmittance, and the farther away of the worker Process so that the transmittance is higher as it is on the side, and transmit to the head-mounted display side through each communication unit,
When the head mounted display receives 3D model image data of the processed safety fence, the virtual fence display system projects the 3D model image data on the display. The position / direction information acquisition unit is arranged on a head-mounted display so as to capture an image in the front direction of the operator's head; and
The image processing unit configured to acquire information on a position and a direction of the worker from an image of the robot main body included in the image data when the image data captured by the image pickup device is acquired. The virtual fence display system according to 1.   The virtual fence display system according to claim 1, wherein the image processing unit changes the transmittance of the 3D model image data in a stepped manner.

Images