WO2018145133A1

WO2018145133A1 - A supervisory, diagnostic and technical assistance device and a system for promoting remote supervision, diagnostic and technical assistance

Info

Publication number: WO2018145133A1
Application number: PCT/ZA2018/050008
Authority: WO
Inventors: Pieter Jacobus Adriaan RAUBENHEIMER
Original assignee: Raubenheimer Pieter Jacobus Adriaan
Priority date: 2017-02-02
Filing date: 2018-02-02
Publication date: 2018-08-09

Abstract

A device for providing supervisory, diagnostic or technical assistance, the device including a user-wearable item with a communication module, a control unit, an actuator, an adjustable light source, a camera and a power supply pack attached to the item wherein the control unit is responsive to commands from the communication module which is in wireless communication with a remote control centre, under the supervision of a skilled technician, so that information/signals/commands are relayed between the communication module and the remote control centre to control the actuator to produce a user- discernible signal and a direction of a unidirectional beam of the light source relative to the item thereby to guide a user wearing the item.

Description

A SUPERVISORY, DIAGNOSTIC AND TECHNICAL ASSISTANCE DEVICE AND A SYSTEM FOR PROMOTING REMOTE SUPERVISION, DIAGNOSTIC AND

TECHNICAL ASSISTANCE

BACKGROUND OF THE INVENTION

[0001] This invention relates to a supervisory, diagnostic and technical assistance device and a system for providing supervisory, diagnostic and technical assistance.

[0002] For example in the mining industry, complex machinery is often employed. A site at which the machinery is employed may be remote from sophisticated services and resources. When a machine operating on a mine breaks down, a skilled engineer/technician is often required to service or repair the machine, for an operator of the machine at the installation site may lack the requisite training or technical knowledge to service or repair the machine in a competent manner.

[0003] There are other industries (for example production, retail, manufacturing or food supply) where a remote office, site or shop needs to be supported or guided by a technician or supervisor from head-office.

[0004] The technician or supervisor may be called upon to travel from one location to another remote location to attend to a minor problem that caused the breakdown of the machine or to inspect the machine or premises. This is expensive and wastes time and resources. It is considered that by providing the operator with suitable guidance, minor problems could be diagnosed and solved by the operator himself before deciding whether it is necessary to incur the expense of calling for the services, and hence the presence, of the technician. [0005] It is an aim of the current invention to provide a device and a system to allow a technician or supervisor to provide remote guidance to an operator to inspect, assess, service or repair an object e.g. a machine.

SUMMARY OF INVENTION

[0006] The invention provides a device for providing supervisory, diagnostic or technical assistance, the device including a user-wearable item and, attached to the item: a) a communication module; b) a control unit; c) an actuator; d) at least an adjustable light source, which is configured to emit a substantially unidirectional light beam; e) a camera configured to record and relay a video feed of a field of view; and f) a power supply pack to power at least the communication module, the control unit, the first actuator, the first light source and the camera; wherein the control unit is responsive to commands from the communication module to control, either directly or indirectly, at least: g) the actuator to produce at least one type of user-discernible signal; and h) a direction of the unidirectional beam of the light source relative to the item. [0007] The communication module may be in wireless communication with a remote control centre which is under the supervision of a skilled technician. Information/signals/commands are relayed between the communication module and the remote control centre. [0008] The user-discernible signal produced by the actuator and the direction of the unidirectional beam may be configured to allow the skilled technician to direct or draw the user's attention in a particular direction.

[0009] The actuator may be of any suitable kind such as a vibrator, a heating pad, and a cam and motor arrangement, etc. [0010] Preferably use is made of a plurality of actuators, each mounted to the item in such a way that, in use, each actuator is in direct or indirect contact with any appropriate part of the user's body.

[0011] The type of user-discernible signal may correspond to a respective actuator, i.e. the signal may be a vibration, a heat sensation or a mechanical force e.g. pushing and/or pulling, which is felt on the body part.

[0012] The light source may be a laser. The laser, in response to a signal from the control centre via the communication module, may be operable to convey/communicate predefined information to the user. The information may be communicated by means of flashing of the unidirectional beam, colour changing of the unidirectional beam or both.

[0013] Alternatively or additionally, the first light source may include a projector which, in response to a signal from the control centre via the communication module, projects an image into or onto the field of view to communicate particular information to the user. [0014] An illuminator may be attached to the item, at an appropriate location, to illuminate the object or a surrounding environment presented to the user.

[0015] A display device may be attached to the item at a location where the user can view information displayed thereon. The display device may be configured to receive a suitable signal from the control centre via the communication module to communicate information to the user.

[0016] The camera may be mounted to the item via a mechanism which is responsive to a signal from the control centre, via the communication module, to control an aim of the camera or a field of view (e.g. to zoom in and to zoom out). The camera may be capable of providing a continuous video feed of the field of view to the control centre via the communication module.

[0017] The field of view may typically be of a subject presented to a user.

[0018] The subject may include an object under review and its surroundings. The object may be a machine or apparatus which requires remedial work, repair, inspection, assessment, calibration, or the like.

[0019] The device may include a microphone and loudspeaker set, the microphone being configured to provide an audio feed of the subject to the control centre via the communication module and the speaker being configured to provide an audible signal from the control centre. [0020] The device may include, directly or indirectly linked accessories. The microphone and loudspeaker set may be mounted, for example, in a Bluetooth headset which is in communication with the communication module, enabling direct communication between the control centre and the user. [0021] A cable, connectable to a suitable port located on or in the object e.g. in the nature of a diagnostic port, may be connected to the communication module to relay information to the control centre. The relayed information may include diagnostic information relating to an error or malfunction etc. in the object or software which drives the object.

[0022] The item may be a garment, hat or a helmet.

[0023] The invention also provides a supervisory diagnostic and assistive system suitable for guidance of a body or part of a body of an operator by a skilled technician, which system includes; a) a first communication system, b) a second communication system; c) the first and second communication systems being capable of wireless communication with each other; d) a support, wearable on the body or on a part of the body of the operator, on which at least some components of the first communication system are mounted; wherein the first communication system includes, at least, the following; e) a power source for supplying power to components of the first communication system; f) a first modem for establishing wireless communication between the first and the second communication systems; g) a camera for obtaining visual input which is wirelessly communicated to the second communication system; h) at least one communication device suitable for receiving input from the second communication system and providing sensorial output to the operator; and i) a first control unit for processing input from the second communication system and input from the camera; and wherein the second communication system includes, at least, the following components; j) a second modem for establishing wireless communication with the first communication system; k) a controller for providing input from the second communication system to the first communication system;

I) a first display for providing visual output, received from the first communication system; and m) a second control unit for processing input from the controller and output to the first display.

[0024] A mechanism may be included to control the field of view and the aim of the camera. Visual input supplied by the camera may include images or a video feed of a subject which is presented to the operator. The subject may include an object, such as a machine or apparatus, and its surroundings. This is exemplary only and non-limiting.

[0025] The communication device may include any one or a combination of the following; a) a first display device;

an illuminating light source;

c) first and second actuators;

a directional light source; and

e) a data transfer cable.

[0026] Each communication device may receive information/instructions from the control unit via a signal transmitted from one modem to the other modem.

[0027] The support may for example be in the form of a jacket or a vest or a hat or a helmet. [0028] The display device may be mounted to the support and may be in the form of a touchscreen.

[0029] The illuminating light source may be mounted to a front of the jacket, vest or helmet and may include one or more LEDS.

[0030] The first and second actuators may each be located in or on a respective sleeve of the jacket or in a crown or band of the hat or helmet (as the case may be) and may each be capable of producing vibrations in response to a distinct signal received from the respective modem.

[0031] The directional light source may be a laser. The laser may be mounted to a shoulder of the jacket or to a crown of the hat or helmet by means of an adjustable mechanism to adjust a direction of a beam emitted from the laser. The mechanism and thus the direction of the beam may be controlled by the second communication system via the modem. [0032] The data transfer cable may be connected to the first modem and may be configured to be connected to the object, e.g. plugged into a suitable port located in or on the object.

[0033] In the second communication system, the visual output may relate to the visual input supplied by the camera of the first communication system. The visual output is interpreted by a technician who may have access to a schematic or a representation of the object which the operator is to service or repair, and which is at a location remote from the technician. Based on what is displayed, the technician may operate the controller to provide input to the first communication system via communication between the first and second modems.

[0034] The controller may be of any suitable kind including a keyboard or a joystick. The operator may input information using the controller which is then processed by the second control unit and transmitted via the modem to the second communication system.

[0035] The first communication system may include an accessory such as a head set which is connected to the first modem. The head set may comprise a loudspeaker and a microphone to enable direct communication between the technician and the operator. The communication between the operator and the technician may be audio communication or in text or video form.

BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The invention is further described by way of example with reference to the accompanying drawings in which:

Figure 1 shows a device according to the invention; Figure 2 is a schematic representation of a system according to the invention. DESCRIPTION OF PREFERRED EMBODIMENT

[0037] Figure 1 shows a device 10 according to the invention which includes a garment in the form of a jacket 12 and, attached to the garment, a communication module 14, a control unit 16, a first actuator 18 fixed to a sleeve 20 of the garment/jacket and a second actuator 22 fixed to a sleeve 24 of the garment/jacket. A laser or projector 26, coupled to two servo motors 28 is attached to a shoulder of the jacket 12. A battery pack 30 is mounted to a back of the jacket and supplies power to the communication module 14, the control unit 16, the first and second actuators 22, the servo motors 28 and the laser 26.

[0038] A camera 32 and an LED light source 34 are mounted on a front of the jacket and are also powered by the battery pack 30.

[0039] A touchscreen 36 is mounted on the sleeve 20.

[0040] A data transfer cable, not shown, is connected to the communication module 14. A free end of the cable may, for example, be connected to a diagnostic port of an object which is to be reviewed and which requires technical assistance.

[0041] The device 10 forms a part of a first communication system 38 in communication with a remote communication system 40 which includes a communication unit 42 capable of bi-directional communication with a processor 44. Suitable software 46 is loaded onto the processor 44. A controller 48, a display 50 and a microphone 52 are connected to the processor 44.

[0042] In use an operator 54 in attendance at a machine 56 (i.e. the object referred to hereinafter) which typically is at a remote location relative to the second communication system 40, draws on the jacket 12 by placing a first arm through the first sleeve 20 and a second arm though the second sleeve 24 so that the actuators 8 and 22 are in close contact with the respective arms of the operator.

[0043] The camera 32 is in use located on the operator's chest and produces a video feed or images of the operator field of view (including the machine 56 and surroundings of the machine 56). The light source 34, also located on the chest of the operator, illuminates a scene that is recorded by the camera 32. A video feed or images produced by the camera 32 are communicated via the communication module 14 to the communication unit 42 and are then displayed on the display 50 to an offsite skilled technician 58 who, if necessary, has access to a schematic or representation 60 of the machine 56. In response to what the technician sees, the technician uses the controller 48 to send commands, via the communication module 14 and the unit 42 to the first communication system 38. The commands could vary depending on what the technician 58 wants the operator 54 to do. [0044] A command could signal a direction in which the operator needs to look or go. This could be achieved by actuating a selected one of the actuators 18 and 22 to produce a user-discernible signal, such as a vibration (for example). The vibration is felt on the operator's arm which is in contact with the respective actuator indicating to the operator that the technician wants the operator to go/look/move in a direction of the arm or to inspect that side of the machine that felt the vibration. The actuators could be of any suitable type and could produce any type of user-discernible signal. The invention is not limited in this respect.

[0045] A command may redirect the camera 32 to change the field of view (zoom in/zoom out) presented on the display 50, giving the technician 58 a better perspective of the surroundings. [0046] A command may direct a beam (not shown) of the laser 26 by actuating the servo motors 28 to focus the beam on a particular item or component of the machine thereby to draw the operator's attention thereto.

[0047] Instructional information, in audio or visual form, can be communicated to the operator via a headset 62 (which includes a microphone and a loudspeaker or the touchscreen 36, respectively.

[0048] A data cable 64 is connected into a suitable port located in the machine 56 to transmit diagnostic information to the second communication system which is interpreted by the software 46 and a suitable update or diagnostic test can then be done automatically.

[0049] The use of a jacket, as described, is non-limiting and it is envisaged that the components, described, could be adapted to be fitted onto a vest, hat or helmet and still provide the necessary functions.

[0050] The device 0 and the system of the invention provide a substantially cheaper solution than using full robotics, remotely operated by a technician, to service and repair a broken machine. The solution could potentially be more cost effective than requiring the technician to travel to a remote location at which the machine is located.

[0051] The invention could also be used in alternative applications, e.g. an employer could provide training supervision or guidance to an employee at a remote office, site or shop.

Claims

1 ) A device for providing supervisory, diagnostic or technical assistance, the device including a user-wearable item and, attached to the item: a) a communication module; b) a control unit; c) an actuator; d) an adjustable light source, which is configured to emit a substantially unidirectional light beam; e) a camera configured to record and relay a video feed of a field of view; and f) and a power supply pack to power at least the communication module, the control unit, the first actuator, the first light source and the camera; wherein the control unit is responsive to commands from the communication module to control, either directly or indirectly, at least: g) the first actuator to produce at least one type of user-discernible signal; and h) a direction of the unidirectional beam of the light source relative to the item.

2. A device according to claim 1 wherein the communication module is in wireless communication with a remote control centre, which is operable under the supervision of a skilled technician.

3. A device according to claim 1 or claim 2 wherein the actuators are selected from the following; a vibrator, a heating pads, and a cam and motor arrangement.

4. A device according to claim 3 wherein use is made of a plurality of actuators, each mounted to the item in such a way that, in use, each actuator is in direct or indirect contact with an appropriate part of the user's body.

5. A device according to claim 4 wherein each type of user-discernible signal corresponds to a respective actuator such a vibration, a heat sensation or a mechanical force, which is felt on the body part.

6. A device according to any one of claims 1 to 5 wherein the light source is a laser.

7. A device according to any one of claims 1 to 6 wherein the light source, in response to a signal from the control centre via the communication module, projects an image into or onto the field of view to communicate particular information to the user.

8. A device according to any one of claims 1 to 7 wherein a display device is attached to the item at a location where the user can view information displayed thereon and the display device is configured to receive a suitable signal from the control centre via the communication module to communicate information to the user.

9. A device according to any one of claims 1 to 8 wherein the camera is mounted to, what in use, is a front or the shoulder of the garment, or the top or side of the hat or helmet, via a mechanism which is responsive to a signal from the communication module, to control an aim or field of view of the camera.

10. A device according to claim 9 wherein the field of view is of a subject presented to the user.

1 1. A device according to claim 10 wherein the subject includes an object under review and its surroundings.

12. A device according to claim wherein the object is a machine or apparatus which requires inspection, assessment, remedial work, repair, calibration.

13. A device according to any one of claims 1 to 12 which includes a microphone and loudspeaker set, the microphone being configured to provide an audio feed of the subject to the control centre via the communication module and the speaker being configured to provide an audible signal from the control centre.

14. A device according to claim 13 wherein the microphone and loudspeaker set are mounted in a headset which is in communication with the communication module, enabling direct communication between the control centre and the user.

15. A device according to any one of claims 1 to 14 wherein a cable, connectable to a suitable port located on or in the object, is connected to the communication module to relay information to the control centre.

16. A device according to any one of claims 1 to 8 wherein the item is a garment, hat or a helmet.

17. A supervisory diagnostic and assistive system, for use with an object, suitable for guidance of a body or a part of a body of an operator by a skilled technician, which system includes: a) a first communication system, b) a second communication system; c) the first and second communication systems being capable of wireless communication with each other; d) a support, wearable on the body or on a part of the body of the operator, on which at least some components of the first communication system are mounted; wherein the first communication system includes, at least, the following: e) a power source for supplying power to components of the first communication system; f) a first modem for establishing wireless communication between the first and the second communication systems; g) a camera for obtaining visual input which is wirelessly communicated to the second communication system; h) at least one communication device suitable for receiving input from the second communication system and providing sensorial output to the operator; and i) a first control unit for processing input from the second communication system and input from the camera; and wherein the second communication system includes, at least, the following components: j) a second modem for establishing wireless communication with the first communication system; k) a controller for providing input from the second communication system to the first communication system; I) a first display for providing visual output, received from the first communication system; and m) a second control unit for processing input from the controller and output to the first display.

18. A system according to claim 17 which includes a mechanism to control the field of view and the aim of the camera.

19. A system according to claim 17 or 18 wherein the communication device includes any one or a combination of the following;

a) a first display device;

b) an illuminating light source;

c) first and second actuators;

d) a directional light source; and

e) a data transfer cable.

20. A system according to claim 19 wherein the communication device receives information/instructions from the control unit via a signal transmitted from the second modem to the first modem.

21 . A system according to any one of claims 17 to 20 wherein the support is a jacket or a vest or a hat or a helmet.

22. A system according to any one of claims 17 to 21 wherein the display device is mounted to the support and is in the form of a touchscreen.

23. A system according to any one of claims 7 to 22 wherein the illuminating light source is mounted to a front of the jacket, vest or helmet and includes one or more LEDS.

24. A system according to any one of claims 17 to 23 wherein the first and second actuators are located in or on a respective sleeve of the jacket or in a crown or band of the hat or helmet and are capable of producing vibrations in response to a distinct signal received from the first modem.

25. A system according to any one of claims 17 to 24 wherein the directional light source is a laser.

26. A system according to claim 25 wherein the laser is mounted to a shoulder of a jacket or to a crown of a hat or a helmet by means of an adjustable mechanism to adjust a direction of a beam emitted from the laser in response to a signal from the second communication system.

27. A system according to any one of claims 17 to 26 wherein the data transfer cable is connected to the first modem and is configured to be plugged into a suitable port located in or on the object

28. A system according to any one of claims 17 to 27 wherein, in the second communication system, the visual output relates to the visual input supplied by the camera of the first communication system and the controller is operable to provide input to the first communication system via communication between the first and second modems based on what is displayed.

29. A system according to claim 28 wherein the controller is a keyboard or a joystick.

30. A system according to claim 29 wherein the first communication system includes a head set, comprising a loudspeaker and a microphone, which is connected to the first modem.

31. A system according to any one of claims 17 to 30 wherein communication between the operator and the technician is in audio, text or video form.