SE2150115A1 - Method of handling safety in workspace, and industrial system - Google Patents

Abstract

A method of handling safety in a workspace (16) of an industrial system (to) comprising at least one machine (12a-12c), where a human (18) is allowed to enter the workspace (16) during operation of the at least one machine (12a-12c), the method comprising continuously or repeatedly providing operational information (28a-28c) associated with an operation of the at least one machine (12a-12c); and displaying the operational information (28a-28c) by means of an augmented reality, AR, interface (30) associated with the human (18) or by means of a mixed reality, MR, interface (32) associated with the human (18). Also provided is an industrial system (10) comprising at least one machine 12a-12c) in a workspace (16) and a display device (22), where a human (18) is allowed to enter the workspace (16) during operation of the at least one machine (12a-12c).

Description

1O METHOD OF HANDLING SAFETY IN WORKSPACE, ANDINDUSTRLÅL SYSTEM Technical Field The present disclosure generally relates to safety handling in a workspace ofan industrial system. In particular, a method of handling safety in aworkspace of an industrial system comprising at least one machine, where ahuman is allowed to enter the workspace during operation of the at least onemachine; and an industrial system comprising at least one machine in aworkspace and a display device, where a human is allowed to enter the workspace during operation of the at least one machine, are provided.Background An industrial system may comprise one or several industrial robots operatingin a workspace. In some industrial systems, the workspace is enclosed by aphysical fence to protect humans from the one or more industrial robots. Thephysical fence however reduces work efficiency due to the segmentation ofthe workspace caused by the physical fence. Such physical fences also addcosts. The properties of the physical fence also need to be taken into consideration when reprogramming the industrial robot.

As an alternative to a physical fence, some industrial systems comprise amonitoring system. The monitoring system may use various supervisionfunctions to supervise the industrial robots in the workspace to ensure a highsafety level. In case a safety configuration set in the monitoring system isviolated by the industrial robot, the monitoring system can automaticallystop the industrial robot to avoid an accident. The monitoring system mayalso report the violation. An example of such violation is when the industrialrobot moves faster than a maximum speed defined in the monitoring system.A further example of such violation is when a distance between the industrial robot and a human becomes too small, e.g. less than 2 meters. 1O When an external monitoring system has intervened to stop an industrialrobot, the process of getting the industrial robot up and running again mightbe cumbersome. The industrial robot may for example have to be restarted.This affects process quality and production times in the workspace negatively. It is therefore desired to avoid triggering such interventions.

SE 1500299 A1 discloses an industrial robot safety system and a method foravoiding collisions between moving parts of two manipulators, or between amoving part of a manipulator and an object close to the manipulator. Aindustrial system comprises a manipulator and a robot controller forcontrolling the movements and actions of the moving parts of themanipulator. The robot controller comprises a collision avoidance system anda traj ectory planning system. The industrial system further comprises a safetycontroller working in parallel with the collision avoidance system of the robotcontroller. The method comprises different steps of exchanging informationabout the planned trajectory of the moving parts of the respectivemanipulator in order to prevent collisions between the moving parts or between a moving part and an object.Summary One object of the present disclosure is to provide a method of handling safety in a workspace of an industrial system, which method improves safety.

A further object of the present disclosure is to provide a method of handlingsafety in a workspace of an industrial system, which method improves productivity of the industrial system.

A still further object of the present disclosure is to provide a method ofhandling safety in a workspace of an industrial system, which method is less complicated and/ or intuitive.

A still further object of the present disclosure is to provide a method ofhandling safety in a workspace of an industrial system, which method solves several or all of the foregoing objects in combination. 1O A still further object of the present disclosure is to provide an industrialsystem comprising at least one machine in a workspace, which industrial system solves one, several or all of the foregoing objects.

According to one aspect, there is provided a method of handling safety in aworkspace of an industrial system comprising at least one machine, where ahuman is allowed to enter the workspace during operation of the at least onemachine, the method comprising continuously or repeatedly providingoperational information associated with an operation of the at least onemachine; and displaying the operational information by means of anaugmented reality, AR, interface associated with the human or by means of a mixed reality, MR, interface associated with the human.

By using AR or MR to display operational information in this way, safetyincidents of humans working in the vicinity of the machine can be prevented.By means of the method, potential dangers to humans inside the workspacecan be dynamically updated. The method therefore effectively improvessafety of the human-machine collaborative industrial system by means of ARor MR. The method further enables a physical barrier enclosing the workspace to be eliminated.

The method may be used as a complement to an external monitoring systemas required by various standards. Such monitoring system is as suchpreviously known and may for example be used to monitor movements of themachines within the workspace and separation distances between humansand machines within the workspace. The method enables a reduced risk forthe human to trigger a safety reaction by such monitoring system, e.g. due toproximity between the human and the at least one machine. Such safetyreaction may for example comprise imposing a speed limit on the at least one machine and/ or an emergency stop of the at least one machine.

The AR/ MR based method promotes human safety in the workspace while at the same time: 1O - reducing the cost of machine deployment, e.g. due to an eliminated need forcostly physical barriers, leading to a potential decrease of adoption barriers of industrial robots; - the workspace is used more efficiently since the workspace can be more effectively shared with human operators and other machines; and - the production throughput is increased since the machine operation is not frequently slowed down or interrupted.

The method may further comprise issuing a warning based on the operationalinformation. By warning the human early about dangerous situations usingAR/ MR, situations causing a triggering of the monitoring system can beavoided. Consequently, a stopping or reduced speed of the at least onemachine can be avoided and the productivity of the industrial system can be increased.

The operational information may be provided in real-time. Real-time may forexample comprise provision of the operational information at a frequency at Hz or higher, such as at 100 Hz or higher.

The AR interface or the MR interface may be displayed by means of a displaydevice worn by the human. By wearing the display device, the human can seethe workspace including the one or more machines as well as the operationalinformation provided as an overlay by the display device. As an alternative toa display device worn by the human, the AR interface of the MR interface may be integrated into protection equipment worn by the human.

The method may further comprise obtaining environment data of theworkspace by means of at least one environment sensor; and determining theoperational information based on the environment data. Examples of suchenvironment sensor are a camera and a lidar. One or more of the at least one environment sensor may be provided on the display device.

The method may further comprise obtaining movement information of the at least one machine by means of one or more movement trackers attached to 1O the at least one machine; and providing the operational information based onthe movement information. The movement trackers may be either stationary or mobile in the workspace.

The movement trackers may for example comprise motion sensors and/ orfeatures readable by machine vision. Alternatively, or in addition, themovement trackers may comprise transmitters, such as Bluetooth low energybeacons. By means of Bluetooth low energy beacons, the presence and location of various dangers can be signaled in real-time.

The operational information may be provided by a controller of the at leastone machine. One example of such controller is a robot controller of an industrial robot.

The at least one machine may comprise an industrial robot. The industrial robot may or may not be a collaborative robot.

The operational information may comprise information associated with amovement of the industrial robot. Such operational information may forexample comprise a path, a trajectory and/ or a volume occupied by movements of the industrial robot.

The method may further comprise continuously or repeatedly providingconfidence information indicative of a confidence level of the associationbetween the operational information and the operation of the at least onemachine; and communicating the confidence information to the human. Forexample, in case the operational information comprises a future movement ofthe industrial robot, the confidence information may indicate a confidencelevel of such future movement, i.e. a probability that the future movement will be performed by the industrial robot.

The communication of the confidence information to the human maycomprise displaying the confidence information by means of the AR interface or by means of the MR interface. 1O The operational information may comprise one or more danger regions. Byproviding such operational information, the human can be informed ofadditional potential hazards within the workspace. Examples of suchpotential hazards include operation regions of the at least one machine, slippery or uneven surfaces, high temperatures, high voltages and invisible laser radiation. Each danger region may have a unique visual appearance, e.g. a unique color and/ or symbol. The danger regions may for example be visualized as a two-dimensional or a three-dimensional zone.

According to a further aspect, there is provided an industrial systemcomprising at least one machine in a workspace and a display device, where ahuman is allowed to enter the workspace during operation of the at least onemachine; wherein the display device is configured to display operationalinformation associated with an operation of the at least one machine bymeans of an augmented reality, AR, interface associated with the human or by means of a mixed reality, MR, interface associated with the human.The display device may be configured to be worn by the human.

The industrial system may further comprise at least one environment sensorconfigured to obtain environment data of the workspace. In this case, theindustrial system may be configured to determine the operational information based on the environment data.

One or more of the at least one environment sensor may be provided on the display device.

The industrial system may further comprise one or more movement trackersattached to the at least one machine for providing movement information ofthe at least one machine. In this case, the industrial system may beconfigured to provide the operational information based on the movement information. 1O The industrial system may further comprise a controller associated with theat least one machine. In this case, the controller may be configured to provide the operational information.The at least one machine may comprise an industrial robot.

The operational information may comprise information associated with a movement of the industrial robot.

The industrial system may be configured to continuously or repeatedlyprovide confidence information indicative of a confidence level of theassociation between the operational information and the operation of the atleast one machine; and communicate the confidence information to the human.

The display device may be configured to display the confidence information by means of the AR interface or by means of the MR interface.The operational information may comprise one or more danger regions.Brief Description of the Drawings Further details, advantages and aspects of the present disclosure will becomeapparent from the following description taken in conjunction with the drawings, wherein: Fig. 1: schematically represents a human wearing a display device and anindustrial system comprising an industrial robot; Fig. 2: schematically represents the industrial system and operationalinformation of the industrial robot displayed by the display device; Fig. 3: schematically represents the industrial system and furtheroperational information of the industrial robot displayed by thedisplay device; Fig. 4: schematically represents the industrial system and further operational information of the industrial robot and of a high- voltage cabinet displayed by the display device; and 1O Fig. 5: schematically represents the industrial system and furtheroperational information of the industrial robot and of a laser source displayed by the display device.Detailed Description In the following, a method of handling safety in a workspace of an industrialsystem comprising at least one machine, where a human is allowed to enterthe workspace during operation of the at least one machine; and an industrialsystem comprising at least one machine in a workspace and a display device,where a human is allowed to enter the workspace during operation of the atleast one machine, will be described. The same or similar reference numerals will be used to denote the same or similar structural features.

Fig. 1 schematically represents an industrial system 10. The industrial system10 of this example comprises an industrial robot 12a and a monitoring system14. The industrial robot 12a performs task in a workspace 16 under thesupervision of the monitoring system 14. As shown in Fig. 1, a human 18 ispresent inside the workspace 16. The human 18 may perform tasks together with the industrial robot 12a.

The industrial robot 12a is one example of a machine according to the present disclosure. The industrial robot 12a comprises a controller 20.

The industrial system 10 further comprises a display device 22. The displaydevice 22 is here exemplified as a pair of AR/ MR glasses worn by the human18.

The monitoring system 14 of this example comprises a monitoring device 24and a monitoring control system 26. The monitoring device 24 is in signalcommunication with the monitoring control system 26. The monitoringdevice 24 may for example be a lidar. By means of the monitoring device 24,the monitoring system 14 can supervise the workspace 16. For example, incase the human 18 comes too close to the industrial robot 12a or if the industrial robot 12a moves too fast, the monitoring system 14 may command 1O an emergency stop of the industrial robot 12a. Although only one is shown,the monitoring system 14 may comprise a plurality of such monitoring devices 24.

As further shown in Fig. 1, the controller 20 sends operational information28a to the display device 22. The operational information 28a is associatedwith an operation of the industrial robot 12a. The operational information28a here comprises a current movement segment that is currently executedby the industrial robot 12a and a plurality of future movement segments thatare planned to be executed by the industrial robot 12a. The controller 20 sends the operational information 28a to the display device 22 in real-time.

Fig. 2 schematically represents the industrial system 10 and operationalinformation 28a of the industrial robot 12a displayed by the display device22. Fig. 2 shows an AR (augmented reality) interface 30 and an MR (mixedreality) interface 32. The AR interface 30 and the MR interface 32 arepresented to the user by means of the display device 22. The features addedto the display in the display device 22 are shown with dashed lines, dottedlines or dash-dotted lines. By means of the display device 22, the human 18sees the workspace 16 and the operational information 28a presented as an overlay to the workspace 16.

In this specific example, the operational information 28a comprises ageometric path 34a of a trajectory currently executed by the industrial robot12a and a geometric path 34b of a future trajectory to be executed next by theindustrial robot 12a. The paths 34a and 34b are differently visuallyrepresented, e.g. by means of different colors. The prediction of the futuresteps of the trajectory may be based on knowledge from a motion planning program of the industrial robot 12a, as communicated by the controller 20.

The operational information 28a of this example further comprises twodanger regions 36a and 36b. Each danger region 36a and 36b is hereexemplified as a three-dimensional volume around the industrial robot 12a.

The first danger region 36a is smaller and closer to the industrial robot 12a 1O than the second danger region 36b. The first danger region 36a is thus morecritical to safety than the second danger region 36b. The first danger region36a is here a forbidden region for the human 18 and the second danger region 36b is a region where potential dangers can occur.

Each danger region 36a and 36b may be displayed with a uniquevisualization, e.g. with a unique color, indicative of its criticality. The firstdanger region 36a may for example be red and the second danger region 36bmay for example be yellow. A further region (not shown) adding a greenimage filter to an area could be shown to indicate absolute freedom from hazards.

The operational information 28a of this example further comprisesconfidence information 38. The confidence information 38 represents aconfidence level of the operational information 28a. Since the operationalinformation 28a in this example is communicated to the display device 22from the controller 20, the confidence level can reach 100 %. To this end, atext string "100%" is displayed by means of the display device 22 as the confidence information 38.

The paths 34a and 34b, the danger regions 36a and 36b, and the confidenceinformation 38 are here examples of operational information 28a associated with movements of the industrial robot 12a.

By means of the AR interface 30 or the MR interface 32, the human 18becomes more aware of the operational status of the industrial robot 12a. Thehuman 18 can thereby more easily avoid triggering a costly safety intervention by the monitoring system 14.

In addition to a visual display of the operational information 28a by means ofthe display device 22, the display device 22 (or other parts of the industrialsystem 10) may optionally provide audio output and/ or text to conveywarning messages. Advice on safety equipment requirements may also beprovided via the display device 22. Other sensory cues such as vibrations of the display device 22 to indicate potential dangers are also possible. 1O 11 Note that display of the operational information 28a to provide a safetyassessment is performed dynamically in real-time and results in a continuousupdate of the projected surrounding area and superposed information. Onemain difference compared to existing solutions is that safety hazards can bemade visible to the human 18 at an earlier time before the monitoring system 14 intervenes to slow down or stop the industrial robot 12a.

Fig. 3 schematically represents the industrial system 10 and furtheroperational information 28a of the industrial robot 12a, as displayed by thedisplay device 22. Mainly differences with respect to Fig. 2 will be described.In Fig. 3, the display device 22 comprises a plurality of environment sensors40 for motion tracking. The environment sensors 40 may for example becameras or lidars. By means of environment data of the workspace 16, asobtained by the environment sensors 40, the display device 22 can determinethe operational information 28a. Thus, instead of communicating theoperational information 28a from the controller 20 to the display device 22,the display device 22 determines the operational information 28a based onthe environment data. For this reason, the confidence level, as displayed bythe confidence information 38, is typically lower, here exemplified as 90 %.The display device 22 of this example is a completely stand-alone solutionwhere the environment sensors 40 detect poses, speeds and trajectories ofthe industrial robot 12a to display various operational information 28a, such as the current path 34a and the future path 34b.

In addition to trajectories of the industrial robot 12a, the environment dataobtained by the environment sensors 40 can be used to detect other potentialhazards and to warn the human 18. As shown in Fig. 3, the operationalinformation 28a here also comprises a third danger region 36c and a fourthdanger region 36d, detected by the environment sensors 40. The third dangerregion 36c is here exemplified as a slippery region and the fourth dangerregion 36d is here exemplified as a toxic region. The environment sensors 40may alternatively, or in addition, be provided at other positions in the workspace 16 than on the display device 22. 1O 12 Fig. 4 schematically represents the industrial system 10 and furtheroperational information 28b. In addition to the industrial robot 12a, theindustrial system 10 here comprises a high-voltage cabinet 12b. The high-voltage cabinet 12b is a further example of a machine according to the present disclosure.

The display device 22 displays operational information 28b associated withan operation of the high-voltage cabinet 12b, e.g. that a high-voltage ispresent. In this example, a movement tracker 42 in the form of a Bluetoothlow energy beacon is provided on the high-voltage cabinet 12b. Themovement tracker 42 communicates the position of the high-voltage cabinet12b to the display device 22 which displays the operational information 28b,by means of the AR interface 30 or the MR interface 32, as a spheresurrounding the high-voltage cabinet 12b. The human 18 can thereby moreeasily avoid coming too close to the high-voltage cabinet 12b to trigger a potential safety reaction by the monitoring system 14.

The industrial system 10 of the example in Fig. 4 further comprises a pluralityof movement trackers 42. The movement trackers 42 are here exemplified asaccelerometers in signal communication with the display device 22. Onemovement tracker 42 is fixed to each link of the industrial robot 12a. Whenthe industrial robot 12a moves, movement information can be obtained bymeans of the movement trackers 42 to provide the operational information28a for display by means of the display device 22. In this case, the confidencelevel, as shown by the confidence information 38, is for example 95 %. As analternative to accelerometers, the movement trackers 42 may be Bluetoothlow energy beacons or may provide features for detection by image processing, such as QR (quick response) codes.

Fig. 5 schematically represents the industrial system 10 and furtheroperational information 28c. Mainly differences with respect to Fig. 4 will bedescribed. Instead of the high-voltage cabinet 12b, the industrial system 10 inFig. 5 comprises a laser source 12c. The laser source 12c is a further example of a machine according to the present disclosure. 1O 13 The display device 22 displays operational information 28c associated withan operation of the laser source 12c, e.g. that an otherwise invisible laserbeam is emitted. In this example, the operational information 28c iscommunicated from the laser source 12c to the display device 22 whichdisplays the operational information 28c, by means of the AR interface 30 orthe MR interface 32, as a sphere surrounding the laser source 12c. Thehuman 18 can thereby more easily avoid coming too close to the laser source 12c to trigger a potential safety reaction by the monitoring system 14.

While the present disclosure has been described with reference to exemplaryembodiments, it will be appreciated that the present invention is not limitedto what has been described above. For example, it will be appreciated that thedimensions of the parts may be varied as needed. Accordingly, it is intendedthat the present invention may be limited only by the scope of the claims appended hereto.

Claims (1)

1. A method of handling safety in a workspace (16) of an industrial system(10) comprising at least one machine (12a-12c), where a human (18) isallowed to enter the workspace (16) during operation of the at least onemachine (12a-12c), the method comprising: - continuously or repeatedly providing operational information (28a-28c) associated with an operation of the at least one machine (12a-12c);and - displaying the operational information (28a-28c) by means of anaugmented reality, AR, interface (30) associated with the human (18) orby means of a mixed reality, MR, interface (32) associated with the human (18). The method according to claim 1, wherein the AR interface (30) or theMR interface (32) is displayed by means of a display device (22) wornby the human (18). The method according to claim 2, further comprising: - obtaining environment data of the workspace (16) by means of at leastone environment sensor (40); and - determining the operational information (28a-28c) based on the environment data. The method according to claim 3, wherein one or more of the at least one environment sensor (40) is provided on the display device (22). The method according to any of the preceding claims, furthercomprising: - obtaining movement information of the at least one machine (12a-12c)by means of one or more movement trackers (42) attached to the atleast one machine (12a-12c); and - providing the operational information (28a-28c) based on the movement information. 1O 10. 11. 12. The method according to any of the preceding claims, wherein theoperational information (28a-28c) is provided by a controller (20) of the at least one machine (12a-12c). The method according to any of the preceding claims, wherein the at least one machine (12a-12c) comprises an industrial robot (12a). The method according to claim 7, wherein the operational information(28a) comprises information associated with a movement of the industrial robot (12a). The method according to any of the preceding claims, furthercomprising: - continuously or repeatedly providing confidence information (38)indicative of a confidence level of the association between theoperational information (28a-28c) and the operation of the at least onemachine (12a-12c); and - communicating the confidence information (38) to the human (18). The method according to claim 9, wherein the communication of theconfidence information (38) to the human (18) comprises displaying theconfidence information (38) by means of the AR interface (30) or by means of the MR interface (32). The method according to any of the preceding claims, wherein theoperational information (28a-28c) comprises one or more danger regions (36a-36d). An industrial system (10) comprising at least one machine (12a-12c) in aworkspace (16) and a display device (22), where a human (18) is allowedto enter the workspace (16) during operation of the at least one machine(12a-12c); wherein the display device (22) is configured to display operationalinformation (28a-28c) associated with an operation of the at least one machine (12a-12c) by means of an augmented reality, AR, interface (30) 1O 13. 14. 15. 16. 17. 18. 19. 16 associated with the human (18) or by means of a mixed reality, MR, interface (32) associated with the human (18). The industrial system (10) according to claim 12, wherein the display device (22) is configured to be worn by the human (18). The industrial system (10) according to claim 12 or 13, furthercomprising at least one environment sensor (40) configured to obtainenvironment data of the workspace (16), wherein the industrial system(10) is configured to determine the operational information (28a-28c) based on the environment data. The industrial system (10) according to any of claims 12 to 14, whereinone or more of the at least one environment sensor (40) is provided on the display device (22). The industrial system (10) according to any of claims 12 to 15, furthercomprising one or more movement trackers (42) attached to the at leastone machine (12a-12c) for providing movement information of the atleast one machine (12a-12c), wherein the industrial system (10) isconfigured to provide the operational information (28a-28c) based on the movement information. The industrial system (10) according to any of claims 12 to 16, furthercomprising a controller (20) associated with the at least one machine(12a-12c), wherein the controller (20) is configured to provide the operational information (28a-28c). The industrial system (10) according to any of claims 12 to 17, wherein the at least one machine (12a-12c) comprises an industrial robot (12a). The industrial system (10) according to any of claims 12 to 18, whereinthe operational information (28a) comprises information associated with a movement of the industrial robot (12a). 1O 20. 21. 22. 17 The industrial system (10) according to any of claims 12 to 19, whereinthe industrial system (10) is configured to: - continuously or repeatedly provide confidence information (38)indicative of a confidence level of the association between theoperational information (28a-28c) and the operation of the at least onemachine (12a-12c); and - communicate the confidence information (38) to the human (18). The industrial system (10) according to claim 20, wherein the displaydevice (22) is configured to display the confidence information (38) by means of the AR interface (30) or by means of the MR interface (32). The industrial system (10) according to any of claims 12 to 21, whereinthe operational information (28a-28c) comprises one or more danger regions (36a-36d).