WO2022258343A1 - Système d'assistance audiovisuelle, procédé et programme informatique pour supporter des travaux de maintenance, des travaux de réparation ou des travaux d'installation dans un système industriel - Google Patents

Système d'assistance audiovisuelle, procédé et programme informatique pour supporter des travaux de maintenance, des travaux de réparation ou des travaux d'installation dans un système industriel Download PDF

Info

Publication number
WO2022258343A1
WO2022258343A1 PCT/EP2022/063632 EP2022063632W WO2022258343A1 WO 2022258343 A1 WO2022258343 A1 WO 2022258343A1 EP 2022063632 W EP2022063632 W EP 2022063632W WO 2022258343 A1 WO2022258343 A1 WO 2022258343A1
Authority
WO
WIPO (PCT)
Prior art keywords
mobile device
maintenance
information
assistance system
output device
Prior art date
Application number
PCT/EP2022/063632
Other languages
German (de)
English (en)
Inventor
Mark Haverkamp
Marco KOEPE
Sebastian Kemper
Robin BRUNNER
Michael STRÖDER
Aram AVAGYAN
Original Assignee
Sms Group Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sms Group Gmbh filed Critical Sms Group Gmbh
Publication of WO2022258343A1 publication Critical patent/WO2022258343A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/20—Administration of product repair or maintenance
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/14—Systems for two-way working
    • H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
    • H04N7/147—Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/32—Operator till task planning
    • G05B2219/32007—Operator is assisted by expert system for advice and delegation of tasks

Definitions

  • Audiovisual assistance system for supporting maintenance work, repair work or installation work in an industrial plant
  • the invention relates to an audiovisual assistance system to support maintenance work, repair work or installation work in an industrial plant, a method for audiovisual support of a maintenance worker by an external expert during maintenance work, repair work or installation work in an industrial plant, and a computer program for executing the method according to the invention.
  • the invention relates to the audiovisual support of maintenance work, repair work or installation work in an industrial plant, in particular a metallurgical production plant for the production of industrial goods such as metallic semi-finished products and/or metallic end products.
  • the industrial plant is used, for example, to produce steel, non-ferrous metals, slabs, blocks, rods, strips, sheets, tubes, beams, forgings or the like.
  • non-ferrous metals are aluminum, copper, nickel, or the like.
  • the production plant is subject to not inconsiderable wear and tear during the manufacture of the products mentioned, for example due to the necessary process temperatures, the forces required during manufacture and the like. Furthermore, as in any technical system, there may be disruptions due to failures, malfunctions or the like of individual machines, parts or components of the production system. For these reasons, maintenance work and/or repair work must be carried out in production plants at regular intervals and at irregular intervals due to incidents. To carry out the maintenance work, repair work or installation work, the maintenance worker goes to the location within the production plant where the maintenance work, repair work or installation work is to be carried out. This is usually a machine, a specific part or component of the production facility, where the production facility extends over a large area.
  • the on-site maintenance worker is equipped with a mobile device that is connected to a stationary computer via a network.
  • Maintenance-related information from a plant information system can be displayed to the maintenance technician on site via a graphic output device.
  • the graphical output device is designed as an augmented reality (AR) display device, so that the information is displayed to the maintenance technician superimposed on reality.
  • AR augmented reality
  • the disclosed assistance system has, for example, means for recognizing components, parts or machines of the production plant. Furthermore, the maintenance worker on site in the
  • US 2007/0273610 A1 discloses a system and a method for displaying maintenance-related information and instructions for a maintenance worker in a system to be maintained.
  • the maintenance technician is shown the information via an augmented reality display.
  • the maintenance-relevant information is stored on a mobile device that the maintenance technician carries with them, for example as a backpack PC. Communication with an external expert is not planned. Comparable systems are known from US 2013/0288719 A1 and WO 2015/028978 A1.
  • EP 1 647 869 A2 discloses an automation system which shows received sensor data on a head-mounted display, in particular an augmented reality system, of a person in the system. Proceeding from this prior art, the invention is based on the object of better supporting the execution of maintenance work, repair work or installation work by a maintenance worker in an industrial plant by an external expert.
  • an audiovisual assistance system for supporting maintenance work, repair work or installation work in an industrial plant, comprising: at least one mobile device for a maintenance worker in the industrial plant, with a first graphic output device for displaying information regarding activities to be carried out by the Maintenance technician and at least one camera for capturing the surroundings of the mobile device, in particular the maintenance technician's field of vision, at least one computer for an external expert with regard to carrying out work in the industrial plant, with a second graphic output device to display the surroundings of the mobile device captured by the camera of the mobile device, in particular the field of view of the maintenance technician; and a network via which the at least one mobile device and the at least one computer can exchange information, in particular data; which is characterized in that the at least one computer comprises a device for detecting and tracking the position and movement of the hands of the external expert, with the detected position and movement of the hands of the external expert being transmitted to the at least one mobile device of the maintenance engineer and the Maintenance worker is displayed by means of the first graphical output device of at least one mobile device.
  • the invention is based on the finding that the execution of maintenance work, repair work or installation work in an industrial plant, in particular an industrial plant such as metallurgical production plants for the production of industrial goods such as metallic semi-finished products and/or metallic end products, often includes many different and complex work steps. Under certain circumstances, complicated activities and work must be carried out, which can only be described in a cumbersome manner.
  • the audiovisual assistance system detects the position and movement of the hands of the external expert and shows the on-site maintenance technician the recorded position and movements of the hands of the external expert.
  • the audiovisual system according to the invention is used in particular to support maintenance work, repair work or installation work in an industrial plant, that is to say generally for work in the industrial plant during its construction and operation.
  • a maintenance worker within the meaning of the invention can also be entrusted with installation work when setting up the industrial plant and is not limited to maintenance work or repair work.
  • the present invention is not limited to overlaying the hands of the external expert, rather this is the minimum requirement.
  • a virtual image of the entire external expert can also be displayed on the first graphic output device of the maintenance technician's at least one mobile device, for example in the form of a so-called avatar.
  • the avatar's hands are then displayed and moved according to the detected position and movements of the external expert's hands.
  • the external expert's computer is designed, for example, as a stationary computer.
  • the recorded position and movement of the hands of the external expert is displayed using the second graphic output device of the at least one computer.
  • the external expert thus gets the recorded position and movements of his hands displayed on the graphic output device of his computer. This allows him to check the recorded position and movements of his hands and, if necessary, adjust them in order to better explain the activities to be carried out to the maintenance technician on site.
  • the audiovisual assistance system according to the invention also includes a plant information system with information about the industrial plant, in particular with information relevant to maintenance, repair or installation.
  • drawings in particular exploded drawings, models, in particular exploded models in 2D or 3D, site plans, maintenance information, information on machines, parts or components of the industrial plant, documents for carrying out maintenance work and/or damage repair or comparable information are stored in the plant information system.
  • the plant information system can be reached via the network, for example.
  • information from the plant information system can also be stored on the mobile device of the maintenance engineer in order to reduce the amount of data to be transmitted and the associated transmission times.
  • the information from the plant information system is displayed to the maintenance technician and/or external experts using the first graphical output device or the second graphical output device.
  • the on-site maintenance worker and/or the external expert has a large number of different pieces of information relating to the industrial plant, including its machines, parts and components, available and can be used when carrying out the maintenance work, repair work or installation work.
  • hidden components, parts or machines of the industrial plant can be displayed.
  • the first graphic output device of the at least one mobile device is designed as a head-mounted display, optical see-through head-mounted display, augmented reality head-mounted display, virtual reality head-mounted Display or a combination thereof formed.
  • the information in particular the position and movements of the hands of the external expert, can be displayed virtually in the maintenance technician's field of vision on site.
  • an overlay with reality as so-called augmented reality has proven to be particularly advantageous proven.
  • useful information from the plant information system can also be displayed for the maintenance engineer on site.
  • a head-mounted display also has the advantage that the maintenance technician has both hands free to carry out activities.
  • the first graphic output device can also be part of the mobile device, for example the display of a tablet or similar mobile device.
  • the audiovisual assistance system in particular the at least one mobile device, includes a device for determining the position of the at least one mobile device in the industrial plant.
  • the device can be used to determine the position of the at least one mobile device and thus also the position of the maintenance worker within the industrial plant.
  • the device is preferably suitable for position determination in an outdoor area and an indoor area and is based, for example, on a GPS system, sensor network or the like. By determining the position, information that is particularly interesting locally can be displayed to the maintenance technician, for example in combination with the plant information system.
  • the audiovisual assistance system in particular the at least one mobile device, includes a device for determining the line of sight of the maintenance worker who is using a mobile device.
  • a device for determining the line of sight of the maintenance engineer By determining the line of sight of the maintenance engineer, the information displayed to the maintenance engineer can be better adapted to the situation.
  • the device for determining the line of sight of the maintenance technician includes, for example, position sensors, a gyro compass, an electronic compass, an image evaluation system or the like.
  • the at least one mobile device includes a device for detecting distances.
  • a device for detecting distances by means of By recording distances, depth information can be generated, i.e. distances to machines, parts or components of the industrial plant.
  • the detected position and movements of the hands of the external expert preferably also include distances or depth information, so that the position and movements of the hands of the external expert displayed to the maintenance worker also take into account the distances or depth information.
  • the external expert can show the maintenance technician on site, preferably in three dimensions, which activities he has to carry out.
  • the device for detecting distances includes, for example, lidar sensors (light detection and ranging), ultrasonic sensors or the like.
  • the area around the mobile device is scanned by means of the device for detecting distances in order to determine the distances to machines, parts or components of the industrial installation located in the area.
  • a grid model (mesh model) is created from this, which represents the environment of the mobile device in three dimensions.
  • the information obtained in this way regarding the surroundings of the mobile device can subsequently be used by the audiovisual assistance system according to the invention, in particular the external expert, to give the maintenance technician more precise instructions, display more relevant information or the like.
  • the information obtained in this way can also be linked to information from other components or systems of the assistance system according to the invention or the industrial plant, such as the plant information system.
  • the second graphic output device of the at least one computer is designed to display a virtual reality, in particular as a virtual reality head-up display or virtual reality cave.
  • the distances recorded by the at least one mobile device can be displayed for the external expert within the virtual reality.
  • a three-dimensional virtual space is thus generated for the external expert, which corresponds to the environment of the maintenance technician on site in the industrial plant.
  • a live image from the maintenance technician's mobile device can be shown to the external expert in the virtual world, so that a kind of augmented reality is generated for the external expert with the situation of the maintenance technician on site in the industrial plant.
  • the audiovisual assistance system in particular the at least one computer, enables the external expert to generate a virtual marking in the industrial installation, in particular in the three-dimensional space of the industrial installation.
  • the virtual marking thus includes three-dimensional location information, in particular as xyz coordinates.
  • the depth information is provided, for example, by the device for detecting distances of the at least one mobile device, or it can be derived from the current position and data from a system model or system plan.
  • the virtual note is displayed to the maintenance engineer and/or the external expert on the first graphical output device or second graphical output device if the maintenance engineer or external expert's field of vision is directed to the area of the industrial plant with the generated virtual note.
  • the external expert can leave notes for the maintenance engineer, who takes them into account when he is at the corresponding position in the industrial plant.
  • the audiovisual assistance system in particular the at least one mobile device, comprises at least one sensor for detecting markers, in particular electronic markers, of the industrial plant, in particular of components, parts and/or machines of the industrial plant. This makes it easy to identify the different components, parts and machines of the industrial plant and, for example, associated information from the
  • the markers and the at least one sensor are based, for example, on an auto ID system, RFID system, barcode system (2D or 3D) or the like.
  • the camera of the mobile device can also be used as a sensor to detect markers of the industrial plant.
  • the at least one mobile device and the at least one computer each comprise audio communication means for audio communication between the maintenance worker and an external expert.
  • the audiovisual assistance system includes a central data store which stores at least the data of the at least one mobile device, in particular of all components involved in the maintenance of the industrial system, for documentation purposes.
  • the audio-visual assistance system thus stores all relevant data that was recorded and exchanged using the system according to the invention.
  • Such documentation is relevant, for example, if an accident occurs in the industrial plant at a later point in time and you want to find the cause of the accident. It can also be used to prove that certain maintenance intervals or the like have been observed.
  • the industrial system in particular machines and components of the industrial system, includes sensors for recording measured values and/or error states, the measured values and/or error states recorded by the sensors being transmitted by the industrial plant, in particular the machines or components of the industrial plant or by control units of the industrial plant, and/or are transmitted to the audiovisual assistance system, in particular to the at least one mobile device, the at least one computer or to a central control system.
  • the recorded measured values are expediently transmitted via the network of the audiovisual assistance system or via another network
  • Error states of the industrial installation can preferably be derived from the recorded measured values. This is done, for example, by the control unit, the central control system or the audiovisual assistance system. In the latter case, by the at least one mobile device and/or by the at least one computer.
  • a control unit is, for example, one or more interconnected ones Programmable logic controllers (PLC or PLCs).
  • PLC or PLCs Programmable logic controllers
  • the multiple programmable logic controllers (PLCs) can be structured hierarchically, with a so-called head PLC collecting and evaluating the data from subordinate PLCs, for example.
  • Fault conditions can be derived from the measured values collected, or the fault conditions can be detected directly using the sensors.
  • the collected measured values or error states can be collected and sent cyclically or sent directly, which is particularly advantageous in the case of serious error states.
  • Error status is sent, for example, in the form of an error code or a more detailed error message.
  • the so-called head PLC collects the sensor data, in particular measured values, from a number of subordinate PLCs or the sensors connected to them.
  • the head PLC evaluates the collected sensor data, for example by means of data correlation and analysis, and based on the evaluation generates an error message, if necessary, which is transmitted to the audiovisual assistance system according to the invention, in particular the maintenance technician's mobile device.
  • the head PLC can also serve as a receiver for information from the mobile device, which is then forwarded to the other PLCs, in particular to adapt the control of the industrial plant.
  • the audiovisual assistance system evaluates the displayed and/or received measured values and/or error states and creates an activity plan for the maintenance worker with the at least a mobile device.
  • the activity plan that has been created is displayed to the maintenance worker using the first graphical output device.
  • the audiovisual assistance system initiates maintenance, repair and/or installation by the on-site maintenance technician, who in turn can call in an external expert as part of the activities to be carried out.
  • the action plan starts from the detected error condition and follows an event tree until the error condition is resolved.
  • the activity plan created is designed as an interactive menu, in particular as a step-by-step guide or according to the question-and-answer principle, and/or the activity plan created is made available to the maintenance worker as a 3D animation and/or displayed as a movie clip.
  • the step-by-step instructions or the question-and-answer principle are particularly designed to be interactive, so that the maintenance worker must confirm or answer the individual steps or questions. For example, measuring a voltage in a switch cabinet can be described as an activity plan.
  • safety instructions are displayed to the maintenance worker by means of the first graphic output device, in particular with regard to the execution of the maintenance work, repair work or installation work.
  • the safety instructions can be general or system-specific.
  • the safety instructions can come from the central control system, be stored in the audiovisual assistance system, or come from other systems of the industrial plant.
  • the safety instructions can also include safety rules or draw attention to special features, such as a required waiting time after switching off a frequency converter for the discharge of existing capacitors or comparable components.
  • the audiovisual assistance system includes a module for ordering spare parts for the industrial system, in particular for machines, components or the like.
  • the maintenance technician and/or the external expert can expediently access the module for ordering spare parts and thus initiate a direct order for spare parts.
  • information from the facility information system, from the device for determining the position of the at least one mobile device and/or the device for determining the line of sight of the maintenance worker can also be taken into account. This additional information makes it easier to identify the spare parts to be ordered and, if necessary, to order them semi-automatically.
  • the module for ordering spare parts automatically suggests the one to be ordered spare part before and the maintenance technician and/or the external expert confirms the suggestion or rejects it or adapts it.
  • spare parts can also be identified using QR codes, component IDs or an e-Doc system.
  • an immediate delivery of spare parts can be arranged.
  • the spare part ordered is delivered directly by a courier or supplier to the maintenance worker on site, for example from a materials warehouse within or in the vicinity of the industrial plant.
  • This is particularly advantageous in the case of time-critical maintenance work or repair work in the event of malfunctions, since this can reduce the downtime of the system.
  • the position of the at least one mobile device, ie the maintenance technician is also transmitted at the same time.
  • the at least one computer is designed to display markings, texts, images, graphics or the like on the first graphic output device of the at least one mobile device, so that the external expert can provide the maintenance technician in the industrial plant with a wide variety of information.
  • the audiovisual assistance system comprises a virtual tool box for displaying virtual tools on the first graphical output device of the at least one mobile device by the external expert.
  • the virtual toolbox provides the tools required for the maintenance work to be carried out, so that the external expert can show them to the maintenance technician on the first graphical output device and show them how to use them.
  • This allows the external expert to show the maintenance engineer the tool to be used on site, so that the maintenance engineer can select it from his real tool box.
  • activities that require the use of a tool can be better displayed. Examples of such activities are measuring a voltage or removing a screwed cover.
  • the device for detecting and tracking the position and movement of the external expert's hands comprises sensors to be worn on the external expert's hands, video sensors, infrared sensors or similar means.
  • the wearable sensors are known, for example, from virtual reality systems with a head-mounted display, where the user holds a controller in both hands, which also include input means or control means in addition to the necessary sensors for position and movement tracking. In principle, however, any type of sensor that is suitable for tracking the position and movement of the hands of the external expert can be used.
  • the network of the audiovisual system according to the invention is expediently designed as a wired and/or wireless network, in particular as a local area network, wireless local area network, mobile network network such as according to the UMTS, LTE or 5G standard, or combinations thereof.
  • the network can also be local or supra-regional, such as an internal local company network or a private or public supra-regional network.
  • the individual transmission technologies and regional extensions can also be combined with one another.
  • the at least one mobile device includes input means in order to adapt the information displayed on the first graphical output device and/or to enter information for display. The maintenance worker can therefore adapt the display on the first graphic output device to his needs and preferences and also enter information which is subsequently displayed on the first graphic output device.
  • the at least one computer comprises input means in order to adapt the information displayed on the first graphic output device and/or the second graphic output device and/or to input information for display.
  • the external expert can therefore input both for the first graphical output device of at least one mobile device and for the second graphical output device of the at least one computer.
  • the external expert can thus display information and/or notes to the maintenance engineer, for example.
  • the audiovisual assistance system comprises a translation system or an interface to a translation system for translating information for the maintenance worker and/or external experts, in particular for translating audio information, texts and/or coded information.
  • a translation system for translating information for the maintenance worker and/or external experts, in particular for translating audio information, texts and/or coded information.
  • the external expert can thus be located anywhere in the world, for example at the headquarters of the manufacturer of the industrial plant, while the maintainer is located at the site of the industrial plant, which is for example in another country or even another continent, and thanks to the translation system external experts and maintenance staff do not have to speak a common language.
  • the translation system can also be used to translate coded information into plain text, such as an error code into a detailed error description.
  • the at least one mobile device is designed as a personal computer, in particular as a backpack PC.
  • the maintenance technician can operate the at least one mobile device using a user interface (desktop) that he is familiar with and does not have to get used to a proprietary user interface.
  • the user interface is in particular the user interfaces known from operating systems such as Windows or MacOS.
  • the at least one mobile device can also use existing software licenses without additional costs or only with lower additional costs.
  • the mobile device designed as a personal computer can be used as a stationary computer and mobile computer, so that the maintenance technician can access his data at any time and the known user interface (desktop) is displayed to him.
  • a backpack PC within the meaning of the invention is a portable PC, regardless of the housing and/or design such as mini/micro housing, laptop, single-board computer, or the like.
  • the essential feature with regard to the present invention is the mobility of the PC, ie its portability for the maintenance worker.
  • the at least one mobile device also includes a documentation module for documenting the maintenance work, repair work or installation work carried out by the maintenance engineer.
  • the documentation module can, for example, automatically record and store the maintenance work, repair work or installation work that has been carried out, which significantly reduces the effort involved for the maintenance worker in this regard.
  • the documentation module can also include an audio interface for voice input.
  • the maintenance worker can thus document the maintenance work, repair work or installation work that has been carried out by means of a simple voice input, in particular at the same time as the execution.
  • voice input allows notes to be entered for later post-processing by a third party.
  • the documentation module can also store images and videos of the maintenance work, repair work or installation work that has been carried out, in particular including the support provided by the external expert. For example, the virtual hands of the external expert, etc., displayed to the maintenance technician, could also be stored.
  • the audiovisual assistance system also includes a virtual reality module, comprising a virtual three-dimensional model of the industrial plant, wherein the at least one mobile device and/or the at least one computer can access the virtual reality module.
  • the maintenance technician and/or the external expert can move through the industrial plant virtually and independently of one another and, for example, create notes, instructions, safety instructions or leave them for the other, which are then displayed.
  • the external expert could be in these different places Leave notes, instructions, safety notices, and when the maintenance worker arrives at the site in the actual industrial plant, the notes, instructions, safety notices are displayed.
  • the virtual reality module is based in particular on a digital twin of the industrial installation and/or on digital twins of machines, parts or components of the industrial installation.
  • a digital twin describes a virtual image of the real system or component and preferably contains all relevant details, information, parameters and the like.
  • the real system or component can expediently be accessed via its associated digital twin.
  • the object is also achieved by a method for the audiovisual support of a maintenance worker by an external expert during maintenance work, repair work or installation work in an industrial plant, comprising the steps: establishing a communication connection between a mobile device of the maintenance worker and a computer of the external expert,
  • the method according to the invention is also based on the knowledge that the execution of maintenance work, repair work or installation work in an industrial plant, in particular an industrial plant such as metallurgical Production plants for the production of industrial goods such as metallic semi-finished products and/or metallic end products, often involving many different and complex work steps. Under certain circumstances, complicated activities and work must be carried out, which can only be described in a cumbersome manner.
  • an industrial plant such as metallurgical Production plants for the production of industrial goods such as metallic semi-finished products and/or metallic end products, often involving many different and complex work steps. Under certain circumstances, complicated activities and work must be carried out, which can only be described in a cumbersome manner.
  • To support a maintenance technician on site who cannot carry out the maintenance work, repair work or installation work independently, but is dependent on the help of an external expert for the execution, it is proposed according to the invention that the position and movement of the hands of the external expert can be adjusted using the method according to the invention recorded and the on-site maintenance technician is shown the recorded position and movements of the hands
  • the method according to the invention serves in particular to support maintenance work, repair work or installation work in an industrial plant, that is to say generally for work in the industrial plant during its construction and operation.
  • a maintenance worker within the meaning of the invention can also be entrusted with installation work when setting up the industrial plant and is not limited to maintenance work or repair work.
  • the present invention is not limited to overlaying the hands of the external expert, rather this is the minimum requirement.
  • the hands can also be a virtual image of the entire external expert on the first graphic output device of the at least one mobile device of the maintenance technician are displayed, for example in the form of a so-called avatar.
  • the avatar's hands are then displayed and moved according to the detected position and movements of the external expert's hands.
  • the external expert is preferably at a stationary location, while the maintenance technician can move freely in the industrial plant, ie is mobile.
  • the method further comprises displaying the detected and tracked position and movement of the hands of the external expert on the second output device of the computer.
  • the external expert thus gets the recorded position and movements of his hands displayed on the graphic output device of his computer. This allows him to check the recorded position and movements of his hands and, if necessary, adjust them in order to better explain the activities to be carried out to the maintenance technician on site.
  • the method according to the invention includes the provision of information about the industrial installation, in particular information relevant to maintenance, repair or installation.
  • the information is provided, for example, by a plant information system and includes drawings, in particular exploded drawings, models, in particular exploded models in 2D or 3D, site plans, maintenance information, information on machines, parts or components of the industrial plant, documents for carrying out maintenance work and/or damage repair or similar Information.
  • the plant information system can be reached via a network, for example.
  • information from the plant information system can also be stored on the mobile device of the maintenance engineer in order to reduce the amount of data to be transmitted and the associated transmission times.
  • the method includes displaying the information about the industrial installation on the first output device of the mobile device and/or the second output device of the computer.
  • the on-site maintenance worker and/or the external expert has a large number of different pieces of information relating to the industrial plant, including its machines, parts and components, available and can be used when carrying out the maintenance work, repair work or installation work.
  • hidden components, parts or machines of the industrial plant can be displayed.
  • the method includes the generation of a virtual reality and/or augmented reality for the maintenance worker by the mobile device, in particular by the first graphic output device of the mobile device, which is a head-mounted display, optical see -Through head-mounted display, augmented reality head-mounted display, virtual reality head-mounted display or a combination thereof.
  • the information in particular the position and movements of the hands of the external expert, can be displayed virtually in the maintenance technician's field of vision on site.
  • an overlay with reality as so-called augmented reality has proven to be particularly advantageous.
  • useful information from the plant information system can also be displayed for the maintenance engineer on site.
  • a head-mounted display also has the advantage that the maintenance technician has both hands free to carry out activities.
  • the information for the maintenance engineer can also be reproduced on a graphical output device of the mobile device, for example on the display of a tablet or similar mobile device.
  • the method includes determining the position of the mobile device in the industrial installation.
  • the determined position of the at least one mobile device also corresponds to the position of the maintenance worker within the industrial installation.
  • the position can be determined in an outdoor area and an indoor area of the industrial installation and is based, for example, on a GPS system, sensor network or the like.
  • the maintenance technician can, in particular, locally Show interesting information, for example in combination with the plant information system.
  • the method includes determining the line of sight of the maintenance worker.
  • the information displayed to the maintenance engineer can be better adapted to the situation.
  • a combination with the determination of the position within the industrial plant and/or the plant information system has proven to be particularly advantageous.
  • the device used to determine the line of sight of the maintenance worker includes, for example, position sensors, a gyro compass, an electronic compass, an image evaluation system or the like.
  • the method includes detecting distances using the mobile device.
  • depth information can be generated, i.e. distances to machines, parts or components of the industrial plant.
  • the detected position and movements of the hands of the external expert preferably also include distances or depth information, so that the position and movements of the hands of the external expert displayed to the maintenance worker also take into account the distances or depth information.
  • the external expert can show the maintenance technician on site, preferably in three dimensions, which activities he has to carry out.
  • the device for detecting distances includes, for example, lidar sensors (light detection and ranging), ultrasonic sensors or the like.
  • the environment of the mobile device is scanned in order to determine the distances to machines, parts or components of the industrial installation located in the environment.
  • a grid model (mesh model) is created from this, which represents the environment of the mobile device in three dimensions.
  • the information obtained in this way regarding the environment of the mobile device can subsequently be used, in particular by the external expert, to give the maintenance technician more precise instructions, display more relevant information or the like.
  • the information obtained in this way can also be linked to information from other components or systems of the assistance system according to the invention or the industrial plant, such as the plant information system.
  • the method according to the invention includes the generation of a virtual reality for the external expert by the computer, in particular by the second graphic output device of the computer, which is designed as a virtual reality head-up display or virtual reality cave.
  • the distances recorded by the maintenance technician's at least one mobile device can be displayed for the external expert.
  • a three-dimensional virtual space is thus generated for the external expert, which corresponds to the environment of the maintenance technician on site in the industrial plant.
  • the external expert can be shown a live image from the maintenance technician's mobile device in the virtual world, so that a kind of augmented reality is generated for the external expert with the situation of the maintenance technician on site in the industrial plant.
  • the method includes the creation of a virtual marking in the industrial installation by the external expert, in particular in the three-dimensional space of the industrial installation.
  • the virtual marking thus includes three-dimensional location information, in particular as xyz coordinates.
  • the depth information is provided, for example, by a device for detecting distances of the at least one mobile device, or can be derived from the current position and data from a system model or system plan.
  • the virtual note is displayed to the maintenance engineer and/or the external expert on the first graphical output device or second graphical output device if the maintenance engineer or external expert's field of vision is directed to the area of the industrial plant with the generated virtual note.
  • the method includes the detection of markers, in particular electronic markers, of the industrial installation, in particular of components and/or machines of the industrial installation.
  • markers in particular electronic markers
  • the markers and the at least one sensor are based, for example, on an auto ID system, RFID system, barcode system (2D or 3D) or the like.
  • these markers can also be captured using the mobile device's camera.
  • the method according to the invention includes the provision of audio communication between the maintenance worker and an external expert.
  • the method includes storing at least the data of the mobile device, in particular all the components involved in the maintenance of the industrial installation, for documentation purposes.
  • the method according to the invention thus stores all relevant data that was recorded and exchanged using the method according to the invention.
  • Such documentation is relevant, for example, if an accident occurs in the industrial plant at a later point in time and you want to find the cause of the accident. It can also be used to prove that certain maintenance intervals or the like have been observed.
  • the method includes the acquisition of measured values and / or error states of the industrial plant, in particular machines and components of the industrial plant, the measured values and / or error states recorded from the industrial plant, in particular the machines or components of the industrial plant or by control units of the industrial plant, and/or are transmitted to the audiovisual assistance system, in particular to the at least one mobile device, the at least one computer or to a central control system.
  • the transmission of the measured values is expediently carried out via the Network of the audiovisual assistance system or another communication network and an interface to the audiovisual assistance system according to the invention.
  • Error states of the industrial installation, in particular machines, parts or components of the industrial installation can preferably be derived from the recorded measured values.
  • a control unit is, for example, one or more interconnected programmable logic controllers (PLC or PLCs).
  • PLC programmable logic controller
  • the multiple programmable logic controllers (PLCs) can be structured hierarchically, with a so-called head PLC collecting and evaluating the data from subordinate PLCs, for example.
  • Fault conditions can be derived from the measured values collected, or the fault conditions can be detected directly using the sensors.
  • the collected measured values or error states can be collected and sent cyclically or sent directly, which is particularly advantageous in the case of serious error states.
  • the error status is sent, for example, in the form of an error code or a more detailed error message.
  • the so-called head PLC collects the sensor data, in particular measured values, from a number of subordinate PLCs or the sensors connected to them.
  • the head PLC evaluates the collected sensor data, for example by means of data correlation and analysis, and based on the evaluation generates an error message, if necessary, which is transmitted to the audiovisual assistance system according to the invention, in particular the maintenance technician's mobile device.
  • the head PLC can also serve as a receiver for information from the mobile device, which is then forwarded to the other PLCs, in particular to adapt the control of the industrial plant.
  • the method according to the invention includes the evaluation of the measured values and/or error states recorded, the creation of an activity plan for the maintenance worker using the mobile device and the display of the created activity plan for the maintenance worker.
  • the evaluation is carried out, for example, by the audiovisual assistance system, in particular the at least one mobile device, or by the industrial installation, in particular a central control system such as a head PLC.
  • the activity plan that has been created is displayed to the maintenance worker using the first graphical output device.
  • the method or the audiovisual assistance system initiates maintenance, repair or installation by the on-site maintenance technician, who in turn can call in an external expert as part of the maintenance work, repair work or installation work.
  • the action plan starts from the detected error condition and follows an event tree until the error condition is resolved.
  • the method includes creating the activity plan as an interactive menu, in particular as a step-by-step guide or according to the question-and-answer principle, and/or displaying the activity plan created as a 3D animation and/or as a film sequence for the
  • step-by-step instructions or the question-and-answer principle are particularly designed to be interactive, so that the maintenance worker must confirm or answer the individual steps or questions. For example, measuring a voltage in a switch cabinet can be described as an activity plan.
  • the method includes the display of safety instructions for the maintenance worker, in particular with regard to the execution of the maintenance work, repair work or installation work.
  • the safety instructions can be general or system-specific. According to the invention, the safety instructions can come from the central control system, be stored in the audiovisual assistance system, or come from other systems of the industrial plant.
  • the safety instructions can also include safety rules or draw attention to special features, such as a required waiting time after switching off a frequency converter for the discharge of existing capacitors or comparable components.
  • the method according to the invention includes ordering spare parts for the industrial plant, in particular for machines, components or the like. The maintenance technician and/or the external expert can expediently initiate the procurement of spare parts. When the order is triggered, information from the
  • Plant information system are taken into account by the position of at least one mobile device and / or the line of sight of the maintenance worker. This additional information makes it easier to identify the spare parts to be ordered and, if necessary, to order them semi-automatically.
  • the procedure automatically proposes the spare part to be ordered and the maintenance technician and/or the external expert confirms the proposal or rejects it or adapts it.
  • spare parts can also be identified using QR codes, component IDs or an e-Doc system. According to a particularly advantageous variant, an immediate
  • the spare part ordered is delivered directly by a courier or supplier to the maintenance worker on site, for example from a materials warehouse within or in the vicinity of the industrial plant.
  • This is advantageous in particular in the case of time-critical maintenance work or repair work in the event of malfunctions, since the downtime of the industrial plant can be reduced as a result.
  • the position of the at least one mobile device, ie the maintenance technician is also transmitted at the same time.
  • the method according to the invention includes the superimposition of markings, texts, images, graphics or the like on the first graphic output device of the mobile device for the maintenance technician by the external expert, so that the external expert can provide the maintenance technician in the industrial plant with a wide variety of information can provide.
  • the method according to the invention includes the provision of a virtual toolbox for displaying virtual
  • the virtual toolbox provides the tools required for the maintenance work to be carried out, so that the external expert can show them to the maintenance technician on the first graphical output device and show them how to use them.
  • This allows the external expert to show the maintenance engineer the tool to be used on site, so that the maintenance engineer can select it from his real tool box.
  • activities that require the use of a tool can be better displayed. Examples of such activities are measuring a voltage or removing a screwed cover.
  • the position and movement of the hands of the external expert are recorded by sensors that can be worn on the hands, video sensors, infrared sensors or comparable means.
  • the wearable sensors are known, for example, from virtual reality systems with a head-mounted display, where the user holds a controller in both hands, which, in addition to the necessary sensors for position and
  • Motion tracking also include input means or control means.
  • input means or control means any type of sensor that is suitable for tracking the position and movement of the hands of the external expert can be used.
  • the method according to the invention expediently includes the transmission of data when executing the method according to the invention via a wired and/or wireless network, in particular via a local area network, wireless local area network, mobile network network such as according to the UMTS, LTE or 5G standard, or combinations thereof.
  • the network can also be local or supra-regional, such as an internal local company network or a private or public supra-regional network.
  • the individual transmission technologies and regional extensions can also be combined with one another.
  • the method includes the adjustment of information displayed on the first graphic output device and/or the input of information for display by the maintenance technician, in particular by means of input means of the mobile device. So the maintainer can adapt the display on the first graphic output device to his needs and preferences and also enter information which is subsequently displayed on the first graphic output device.
  • the method includes the adaptation of information displayed on the first graphic output device and/or the input of information for display by the external expert, in particular by means of input means of the computer.
  • the external expert can therefore make entries both for the first graphic output device of the at least one mobile device and for the second graphic output device of the at least one computer.
  • the external expert can thus display information and/or notes to the maintenance engineer, for example.
  • the method according to the invention includes the translation of information for the maintenance worker and/or external experts, in particular for the translation of audio information, text and/or coded information.
  • the external expert can thus be located anywhere in the world, for example at the headquarters of the manufacturer of the industrial plant, while the maintainer is located at the site of the industrial plant, which is for example in another country or even another continent, and thanks to the translation system external experts and maintenance staff do not have to speak a common language.
  • the translation system can also be used to translate coded information into plain text, such as an error code into a detailed error description.
  • the method for the maintenance technician is carried out using a mobile personal computer, in particular using a backpack PC.
  • the maintenance technician can operate the at least one mobile device using a user interface (desktop) that he is familiar with and does not have to get used to a proprietary user interface.
  • the user interface is in particular the user interfaces known from operating systems such as Windows or MacOS.
  • At least one mobile device can also do this use existing software licenses without additional or only with lower additional costs.
  • the mobile device designed as a personal computer can be used as a stationary computer and mobile computer, so that the maintenance technician can access his data at any time and the known user interface (desktop) is displayed to him.
  • a backpack PC within the meaning of the invention is a portable PC, regardless of the housing and/or design, such as a mini/micro housing, laptop, single-board computer, or the like.
  • the essential feature with regard to the present invention is the mobility of the PC, ie its portability for the maintenance worker.
  • the method includes documenting the maintenance work, repair work or installation work carried out by the maintenance worker, in particular using the mobile device.
  • the documentation can, for example, automatically record and save the maintenance work, repair work or installation work that has been carried out, which significantly reduces the related effort for the maintenance technician.
  • the documentation can also include voice input.
  • the maintenance worker can thus document the maintenance work, repair work or installation work that has been carried out by means of a simple voice input, in particular at the same time as the execution. Furthermore, voice input allows notes to be entered for later post-processing by a third party.
  • the documentation can also include the storage of images and videos of the maintenance work, repair work or installation work carried out, in particular including the support of the external expert. For example, the virtual hands of the external expert, etc., displayed to the maintenance technician, could also be stored.
  • the method includes the provision of a virtual three-dimensional model of the industrial installation, with the maintenance worker and/or the external expert being able to access the virtual three-dimensional model.
  • the maintenance technician and/or the external expert can navigate through the industrial Move the system and, for example, create notes, instructions, safety instructions or leave them for the other, which are displayed below.
  • the external expert could leave notes, instructions, safety notices in these different places, and when the maintainer arrives at the place in the actual industrial plant, the notes, instructions, safety notices are displayed.
  • the method according to the invention is expediently carried out using an audiovisual assistance system according to the invention.
  • the object is also achieved by a computer program comprising instructions which, when the program is executed by a computer, cause the latter to execute the method according to the invention.
  • FIG. 1 is a schematic view of an audiovisual according to the invention
  • FIG. 1 shows a schematic view of an audiovisual assistance system 1 to support maintenance work, repair work or installation work in an industrial plant 11.
  • the inventive audiovisual assistance system 1 includes a mobile device 2 for a maintenance technician 3 in the industrial plant 11.
  • the mobile device 2 for The maintenance technician 3 is equipped with a first graphical output device 4 for displaying information regarding activities to be carried out by the maintenance technician 3 and a camera for capturing the environment of the mobile device 2, in particular the field of view of the maintenance technician 3.
  • the mobile device 2 for the maintenance technician 3 also includes a device 13 for determining the position of the mobile device 2 in the industrial plant 11, a device 14 for determining the viewing direction of the maintenance technician 3 who mobile device 2 uses, a device 15 for detecting distances and audio communication means 16, in particular speakers and a microphone, for example in the form of a headset.
  • the mobile device 1 also includes input means (not shown) in order to adapt the information displayed on the first graphical output device 4 and/or to enter information for display.
  • the mobile device 2 is preferably designed as a personal computer, in particular as a so-called backpack PC.
  • the components assigned to the mobile device 2 of the maintenance technician 3 are summarized in a dashed circle in FIG. 1 .
  • the audiovisual assistance system 1 from FIG. The computer 6 is equipped with a second graphical output device 8 for displaying the surroundings of the mobile device 2 captured by the camera 5 of the mobile device 2 , in particular the field of view of the maintenance technician 3 .
  • the computer 6 of the external expert 7 also includes audio communication means 16, in particular loudspeakers and a microphone, for example in the form of a headset.
  • the computer 6 is preferably designed to overlay markings, texts, images, graphics or the like on the first graphic output device 4 of the mobile device 2 so that the external expert 7 can provide the maintenance technician 3 in the industrial plant 11 with a wide variety of information.
  • the computer 6 includes input means (not shown) in order to adapt the information displayed on the first graphical output device 4 and/or the second graphical output device 8 and/or to enter information for display.
  • the computer 6 comprises a device 10 for detecting and tracking the position and movement of the hands of the external expert 7, the detected position and movement of the hands of the external expert 7 being transmitted to the mobile device 2 of the maintenance technician 3 and to the maintenance technician 3 by means of of the first graphic output device 4 of the at least one mobile device 1 is shown.
  • the detected position and movement of the hands of the external expert 7 are expediently also displayed using the second graphical output device 8 of the computer 6, so that the external expert can check the detected position and movement of his hands.
  • the device 10 for detecting and tracking the position and movement of the hands of the external expert 7 can comprise sensors to be worn on the hands of the external expert 7, video sensors, infrared sensors or comparable means.
  • the network 9 is designed, for example, as a wired and/or wireless network, in particular as a local area network, wireless local area network, mobile network network such as, for example, according to the UMTS, LTE or 5G standard, or combinations from it.
  • the audiovisual assistance system 1 preferably includes a plant information system 12 with information about the industrial plant 11, in particular with maintenance-related information.
  • the plant information system 12 can also be part of the industrial plant 11, which the audiovisual assistance system 1 according to the invention accesses.
  • information from the plant information system 12 can also be stored on the mobile device 2 of the maintenance technician 3 in order to reduce the amount of data to be transmitted and the associated transmission times.
  • the system information system 12 contains, for example, drawings, in particular exploded drawings, models, in particular exploded models in 2D or 3D, site plans, maintenance information, repair information,
  • the plant information system 12 can be reached via the network 9, for example.
  • the information from the plant information system 12 is sent to the maintenance technician 3 and/or external experts 7 using the first graphical output device 4 or the second graphical output device 8 .
  • the first output device 4 of the mobile device 2 is designed as a head-mounted display, in particular as an optical see-through head-mounted display for generating an augmented reality.
  • the maintenance worker 3, who wears the head-mounted display can see his surroundings through it (optical see-through).
  • the position and movement of the hands of the external expert 7 are shown to the maintenance worker by means of the head-mounted display as an overlay with reality, so that an augmented reality is created.
  • safety instructions are preferably additionally displayed to the maintenance technician 3 by means of the first graphic output device 4, in particular with regard to the execution of the maintenance work, repair work or installation work.
  • the first output device 4 can also be part of the mobile device 2, such as the display of a tablet.
  • the second graphic output device 8 of the computer 6 is designed to display a virtual reality, in particular as a virtual reality head-up display or virtual reality cave.
  • the virtual reality shown is based on the recorded surroundings of the mobile device 2, in particular the camera 5.
  • Information from the plant information system 12 can also be displayed.
  • the generated virtual reality contains information from the device 15 for detecting distances, so that the external expert 7 is shown a three-dimensional virtual reality within which he can move his hands, which in turn are shown to the maintenance worker 3 .
  • the audiovisual assistance system 1, in particular the computer 6, preferably enables the external expert 7 to create a virtual marking in the industrial plant 11, in particular in the three-dimensional space of the industrial plant 11.
  • the audiovisual assistance system 1 comprises at least one sensor for detecting markers, in particular electronic markers, of the industrial plant 11, in particular of components, parts and/or machines of the industrial plant 11. In this way, the components Identify parts and/or machines of the industrial plant 11 easily and reliably.
  • the audiovisual assistance system 1 also includes a central data store 17, which stores at least the data of the mobile device 2, in particular all of the components involved in the maintenance, repair or installation of the system 11, for documentation purposes.
  • the audiovisual system 1 preferably stores all exchanged data.
  • the mobile device includes
  • this documentation module can include a voice input in order to make the documentation as simple as possible for maintenance personnel 3 and to reduce the time required for this.
  • the industrial plant 11 from FIG. 1 also includes one or more control units 23, in particular so-called programmable logic controllers (PLCs or PLCs).
  • the multiple programmable logic controllers (PLCs) can be structured hierarchically, with a so-called head PLC collecting and evaluating the data from subordinate PLCs, for example.
  • the multiple control units 23 are preferably connected to one another via a bus system or the like. According to FIG. 1, the industrial installation 11, in particular machines and
  • Measured values and/or error states are displayed by the industrial system 11, in particular the machines or components of the industrial system 11 or by control units 23 of the industrial system 11, and/or to the audiovisual assistance system 1, in particular to the mobile device 2, the computer 6 or to a central control system.
  • the so-called head PLC collects the sensor data, in particular measured values, from several subordinate PLCs or the sensors 18 connected to them.
  • the head PLC evaluates the collected sensor data, for example by means of data correlation and analysis, and generates an error message which is transmitted to the audiovisual assistance system 1 according to the invention, in particular the mobile device 2 of the maintenance worker 3 .
  • the head PLC can also serve as a receiver for information from the mobile device 2, which is then forwarded to the other PLCs, in particular in order to adapt the control of the industrial plant 11.
  • the created activity plan is displayed to the maintenance person 3 by means of the first graphic output device 4 .
  • the activity plan that is created is preferably designed as an interactive menu, in particular as a step-by-step guide or according to the question-and-answer principle. Alternatively or additionally, the created activity plan is displayed to the maintenance worker 3 as a 3D animation and/or as a film sequence.
  • the audiovisual assistance system according to the invention advantageously includes a module 19 for ordering spare parts for the industrial plant 11, in particular for machines, components or the like.
  • the maintenance technician 3 and/or the external expert 7 can access the module 19 for ordering spare parts in order to initiate the procurement of spare parts.
  • the audiovisual assistance system 1 also includes a virtual toolbox 20 for the external expert 7 to display virtual tools on the first graphical output device 4 of the mobile device 2. Using the displayed tools, the external expert 7 can show the maintenance technician 3 in the industrial plant 11 better die activities to be carried out using tools virtually on the first graphical output device 4 show.
  • the audiovisual assistance system 1 includes a translation system 21 for translating information for the maintenance worker 3 and/or external experts 7, in particular for translating audio information, texts and/or coded information.
  • the audiovisual assistance system 1 from FIG. 1 also includes a virtual reality module 22 which contains a virtual three-dimensional model of the industrial plant 11 .
  • Mobile device 2 and computer 6 can access virtual reality module 22 .
  • the maintenance worker 3 and/or the external expert 7 can move through the industrial plant 11 virtually and independently of one another and, for example, create notes, instructions, safety instructions or leave them for the other, which are then displayed.
  • the external expert 7 could leave notes, instructions, safety instructions at these different locations and when the maintenance worker 3 arrives at the location in the actual industrial plant 11, the notes, instructions, safety instructions are displayed on the first graphical output device 4 of the mobile device 2 for the Maintainer 3 displayed.
  • the virtual reality module is based in particular on a digital twin of the industrial installation and/or on digital twins of machines, parts or components of the industrial installation.
  • a digital twin describes a virtual image of the real system or component and preferably contains all relevant details, information, parameters and the like.
  • the real system or component can expediently be accessed via its associated digital twin.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Human Resources & Organizations (AREA)
  • Quality & Reliability (AREA)
  • Marketing (AREA)
  • Manufacturing & Machinery (AREA)
  • Signal Processing (AREA)
  • Economics (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Automation & Control Theory (AREA)
  • Operations Research (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

L'invention concerne un système d'assistance audiovisuelle (1) et un procédé pour supporter des travaux de maintenance, des travaux de réparation ou des travaux d'installation dans un système industriel (11), comprenant : au moins un dispositif mobile (2) pour un membre du personnel de maintenance (3) dans le système industriel (11), avec un premier dispositif de sortie graphique (4) pour afficher des informations relatives à des tâches à effectuer par l'élément de personnel de maintenance (3) et au moins une caméra (5) pour capturer l'environnement autour du dispositif mobile (2), en particulier le champ de vision du membre du personnel de maintenance (3), au moins un ordinateur (6) pour un expert externe (7) pour réaliser des travaux dans le système industriel (11), avec un second dispositif de sortie graphique (8) pour afficher l'environnement autour du dispositif mobile (2) capturé par la caméra (5) du dispositif mobile (2), en particulier le champ de vision du membre du personnel de maintenance (3) ; et un réseau (9) par l'intermédiaire duquel le ou les dispositifs mobiles (2) et le ou les ordinateurs (6) peuvent échanger des informations, en particulier des données ; caractérisé en ce que ledit ou lesdits ordinateur(s) (6) comprend/comprennent un dispositif (10) pour capturer et suivre la position et le mouvement des mains de l'expert externe (7), la position et le mouvement capturés des mains de l'expert externe (7) étant transmis au(x) dispositif(s) mobile(s) (2) du membre du personnel de maintenance (3) et affichés pour le membre du personnel de maintenance (3) au moyen du premier dispositif de sortie graphique (4) du ou des dispositifs mobiles (1).
PCT/EP2022/063632 2021-06-10 2022-05-19 Système d'assistance audiovisuelle, procédé et programme informatique pour supporter des travaux de maintenance, des travaux de réparation ou des travaux d'installation dans un système industriel WO2022258343A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102021205859.0 2021-06-10
DE102021205859.0A DE102021205859A1 (de) 2021-06-10 2021-06-10 Audiovisuelles Assistenzsystem, Verfahren und Computerprogramm zur Unterstützung von Wartungsarbeiten, Reparaturarbeiten oder Installationsarbeiten in einer industriellen Anlage

Publications (1)

Publication Number Publication Date
WO2022258343A1 true WO2022258343A1 (fr) 2022-12-15

Family

ID=82116851

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2022/063632 WO2022258343A1 (fr) 2021-06-10 2022-05-19 Système d'assistance audiovisuelle, procédé et programme informatique pour supporter des travaux de maintenance, des travaux de réparation ou des travaux d'installation dans un système industriel

Country Status (2)

Country Link
DE (1) DE102021205859A1 (fr)
WO (1) WO2022258343A1 (fr)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1647869A2 (fr) 2004-10-15 2006-04-19 Siemens Aktiengesellschaft Transmission de données vers et à partir de composants d'automatisation
US20070273610A1 (en) 2006-05-26 2007-11-29 Itt Manufacturing Enterprises, Inc. System and method to display maintenance and operational instructions of an apparatus using augmented reality
US20130288719A1 (en) 2012-04-27 2013-10-31 Oracle International Corporation Augmented reality for maintenance management, asset management, or real estate management
WO2015028978A1 (fr) 2013-08-29 2015-03-05 Umpi Elettronica - Societa' A Responsabilita' Limitata Procédé et système permettant l'inspection et/ou la maintenance d'un panneau électrique au moyen de la réalité augmentée
DE102016224459A1 (de) * 2015-12-11 2017-06-14 Sms Group Gmbh Mobile visuelle Assistenz in Hütten-, Walz-, Schmiede- und Röhrenwerken
EP3318944A2 (fr) * 2016-11-08 2018-05-09 Rockwell Automation Technologies, Inc. Réalité virtuelle et réalité augmentée pour l'automatisation industrielle
US20180324229A1 (en) * 2017-05-05 2018-11-08 Tsunami VR, Inc. Systems and methods for providing expert assistance from a remote expert to a user operating an augmented reality device
US20190370544A1 (en) * 2017-05-30 2019-12-05 Ptc Inc. Object Initiated Communication

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1647869A2 (fr) 2004-10-15 2006-04-19 Siemens Aktiengesellschaft Transmission de données vers et à partir de composants d'automatisation
US20070273610A1 (en) 2006-05-26 2007-11-29 Itt Manufacturing Enterprises, Inc. System and method to display maintenance and operational instructions of an apparatus using augmented reality
US20130288719A1 (en) 2012-04-27 2013-10-31 Oracle International Corporation Augmented reality for maintenance management, asset management, or real estate management
WO2015028978A1 (fr) 2013-08-29 2015-03-05 Umpi Elettronica - Societa' A Responsabilita' Limitata Procédé et système permettant l'inspection et/ou la maintenance d'un panneau électrique au moyen de la réalité augmentée
DE102016224459A1 (de) * 2015-12-11 2017-06-14 Sms Group Gmbh Mobile visuelle Assistenz in Hütten-, Walz-, Schmiede- und Röhrenwerken
WO2017097902A1 (fr) 2015-12-11 2017-06-15 Sms Group Gmbh Assistance visuelle mobile dans des ateliers de métallurgie, de laminage, de forge et de fabrication de tuyaux
EP3318944A2 (fr) * 2016-11-08 2018-05-09 Rockwell Automation Technologies, Inc. Réalité virtuelle et réalité augmentée pour l'automatisation industrielle
US20180324229A1 (en) * 2017-05-05 2018-11-08 Tsunami VR, Inc. Systems and methods for providing expert assistance from a remote expert to a user operating an augmented reality device
US20190370544A1 (en) * 2017-05-30 2019-12-05 Ptc Inc. Object Initiated Communication

Also Published As

Publication number Publication date
DE102021205859A1 (de) 2022-12-15

Similar Documents

Publication Publication Date Title
DE102006014634B4 (de) Mensch-Maschine-Schnittstelle für ein Kontroll- bzw. Steuerungs-System
DE19953739C2 (de) Einrichtung und Verfahren zur objektorientierten Markierung und Zuordnung von Information zu selektierten technologischen Komponenten
CN106340217B (zh) 基于增强现实技术的制造装备智能系统及其实现方法
EP1163557B1 (fr) Systeme et procede de traitement d'une documentation avec structuration multicouche d'informations, en particulier pour applications techniques et industrielles
EP1451652B1 (fr) Systeme et procede d'etablissement d'une documentation d'operations en vue d'une representation dans un systeme a realite amplifiee
DE102017002603B4 (de) Robotersystem, das Roboter umfassend mehrere mechanische Einheiten steuert, mechanische Einheiten und Robotersteuervorrichtung
EP1157314B1 (fr) Exploitation de technologies fondee sur la realite amplifiee pour l'assistance par des experts eloignes conforme a la situation du travailleur specialise
EP1701233B1 (fr) Génération de réalités virtuelles sur la base d'un environnement réel
DE102019122343A1 (de) Abweichungskorrektur für industrielle anwendungen mit erweiterter realität
EP3337751A1 (fr) Procédé de surveillance d'un chantier, engin de travail et système de surveillance d'un chantier
WO2017097902A1 (fr) Assistance visuelle mobile dans des ateliers de métallurgie, de laminage, de forge et de fabrication de tuyaux
WO2005045729A1 (fr) Systeme et procede pour mettre en oeuvre et visualiser des simulations dans une realite etendue
DE10242007A1 (de) Portabler Computer in einem Prozesssteuerungsumfeld
EP3623891A1 (fr) Hiérarchies d'images pouvant être individualisées pour un système de conduite d'une installation technique
DE102006051533A1 (de) Bedienungshilfe für eine Vorrichtung zum Behandeln von Behältnissen (II)
WO2022258343A1 (fr) Système d'assistance audiovisuelle, procédé et programme informatique pour supporter des travaux de maintenance, des travaux de réparation ou des travaux d'installation dans un système industriel
EP1487616B1 (fr) Commande de processus automatique
DE102004016329A1 (de) System und Verfahren zur Durchführung und Visualisierung von Simulationen in einer erweiterten Realität
DE19546223A1 (de) Verfahren und Managementsystem zum Management von räumlich getrennten Objekten
EP4186651B1 (fr) Système et procédé doté d'un système
EP2515554A2 (fr) Procédé de visualisation de données d'état d'une machine dans une installation ainsi que le dispositif correspondant
EP2703920B1 (fr) Procédé d'apprentissage de la commande d'une machine
WO2024104560A1 (fr) Génération d'hologrammes pour affichage dans un environnement de réalité augmentée
DE102022100840A1 (de) Augmented-reality-system zum unterrichten eines anwenders einer applikation
EP2517079B1 (fr) Dispositif, système et procédé pour faire fonctionner des installations techniques de manière sûre

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22731997

Country of ref document: EP

Kind code of ref document: A1

DPE2 Request for preliminary examination filed before expiration of 19th month from priority date (pct application filed from 20040101)
122 Ep: pct application non-entry in european phase

Ref document number: 22731997

Country of ref document: EP

Kind code of ref document: A1