KR20220010356A - System and method for augmented reality content-based work management - Google Patents

Abstract

According to an embodiment of the present invention, provided are a system and a method for work management based on augmented reality content, and more particularly, a method for remotely managing a field work based on augmented reality content by a work management application executed by a workstation device, which comprises the steps of: building a workstation, and generating a workspace; assigning a task for each field device of a field worker included in the workspace on the basis of augmented reality content; obtaining work data in accordance with the performance of a field work from the field device; generating augmented reality-based management content on the basis of the obtained work data; obtaining sensing data on a work target object of the field work from a field sensor system of a field; generating augmented reality-based sensing content on the basis of the obtained sensing data; setting mapping information for determining a virtual location where the augmented reality content including the management content and the sensing content is to be augmented on an augmented reality screen viewed by the field worker; and performing a remote collaborative communication on the basis of the augmented reality content and the mapping information. Therefore, work safety can increase.

Description

Augmented reality content-based work management system and method {SYSTEM AND METHOD FOR AUGMENTED REALITY CONTENT-BASED WORK MANAGEMENT}

The present invention relates to an augmented reality (AR) content-based task management system and method. More specifically, it relates to an augmented reality content-based task management system and method for providing a remote collaboration service based on augmented reality content.

As information and communication technology (ICT) develops, technology development for exchanging information in real time through a long-distance data communication network is being actively developed.

In particular, as technology fields such as virtual reality (VR) and augmented reality (AR) have recently received attention, interest in a method of exchanging information based on this is a trend.

Along with this trend, the market for services and devices that provide an immersive experience for users is developing day by day.

Thus, in recent years, data or information required for work in various industrial fields is managed based on virtual reality or augmented reality technology, and at the same time, if necessary, based on the above data or information (that is, virtual content) is being communicated.

However, in the conventional virtual environment-based communication, there is a system that not only supports communication between remote users based on the virtual environment, but can also manage and direct the progress of a specific job or industrial site from a macro point of view based on the virtual environment. There is a limit to being incomplete.

In addition, in the prior art field, communication can be performed based on the established virtual content, but activities such as creating, uploading, and sharing new virtual content in real time in a virtual environment-based communication process between remote users are performed. There is a problem in that the effectiveness of virtual environment-based remote communication is lowered due to insufficient supported technology.

In other words, problems in industrial sites such as limitations in management due to the increase of increasingly complex systems in industrial sites, problems of smooth communication between remote workers and managers, and inability to grasp the progress of real-time process work, etc. It is necessary to develop technology to solve the problem.

KR 10-2019-0076034 A

An object of the present invention is to provide an augmented reality content-based task management system and method for implementing a remote collaboration service based on augmented reality content.

In detail, the present invention aims to provide an augmented reality content-based work management system and method for implementing a workstation service capable of managing a user (ie, a worker) and the corresponding work remotely located at the job site. have.

Another object of the present invention is to provide an augmented reality content-based task management system and method for implementing collaborative communication between users (ie, managers and workers) located remotely from each other based on a virtual environment.

Another object of the present invention is to provide an augmented reality content-based task management system and method for generating, uploading, and sharing augmented reality content in real time based on virtual environment-based communication between remote users.

However, the technical problems to be achieved by the present invention and embodiments of the present invention are not limited to the technical problems described above, and other technical problems may exist.

Augmented reality content-based work management system and method according to an embodiment of the present invention is a method for remotely managing field work based on augmented reality content by a work management application executed by a workstation device, building a workstation and creating a workspace; providing tasks for each field device of field workers included in the workspace based on the augmented reality contents; obtaining work data according to the performance of the field work from the field device; generating augmented reality-based management content based on the obtained work data; acquiring sensing data for the work target object of the field work from the field sensor system of the field; generating augmented reality-based sensing content based on the acquired sensing data; setting mapping information for determining a virtual location where the augmented reality content including the management content and the sensing content is to be augmented on the augmented reality screen viewed by the field worker; and performing remote collaborative communication based on the augmented reality content and the mapping information.

In this case, the workstation means a web or mobile service environment that implements a workstation service for remotely managing the field work and the field worker.

In addition, the workstation service provides at least one service of a customized process management service, an augmented reality content link service, a maintenance management service, an augmented reality-based remote collaboration service, and a feed-based process management service.

In addition, the workspace refers to a work organization unit organized by a user of the workstation service by inviting a business person related to the field work on the workstation.

In addition, the step of creating the workspace includes inviting at least one or more field workers as members to the workspace, and allocating a plan for each field worker.

In addition, the plan is information for setting the service use right and service availability period for the workspace.

In addition, the service use right includes at least one of a Make right to produce the augmented reality content, a View right to implement visualization, and a Remote right to perform remote collaborative communication.

In addition, the augmented reality content is content that is augmented in a three-dimensional virtual environment to provide information related to the field work.

In addition, the step of giving the task for each field device includes the step of providing task information for each field device, wherein the task information is information to explain or assist the task assigned to the field worker, and to It includes at least one or more of worker designation information, work target object designation information, work content information, and related augmented reality content information.

In addition, the step of giving the task for each on-site device includes limiting the right to use the augmented reality content related to the task for each on-site device based on the plan.

In addition, the obtaining of the work data may include obtaining augmented reality-based input data from the field device, and operator information about the field worker who has input the input data and the field at the time of input of the input data. and acquiring at least one or more of the operator's location information.

In addition, the management content includes at least one of process management content, maintenance management content, abnormal equipment management content, and feedback management content.

In addition, the acquiring of the sensing data includes acquiring real-time sensing data before, during, or completing the on-site work for the work target object.

In addition, the sensing content is augmented reality content generated by converting the sensing data into the form of an augmented reality-based graph or dashboard.

In addition, the performing of the remote collaborative communication may include providing the augmented reality content for the work target object to the field device of a field worker assigned the task for the work target object, and the augmented reality content Receiving the work data from the field device receiving and updating the augmented reality content based on management history information converted into real content.

In addition, the step of providing the augmented reality content to the field device of the field worker who has been given the task for the work target object, further comprising the step of providing guide information explaining the guidelines for the augmented reality contents do.

In addition, the augmented reality content-based task management system and method according to an embodiment of the present invention, the augmented reality content or the guide information, augmented reality output from the field device based on the mapping information for the augmented reality content The method further includes displaying on the screen.

In this case, the management history information is data generated based on the work data to generate or update the augmented reality content.

In addition, the step of assigning the task for each field device includes dividing the space in which the field work is performed into at least one or more point of interest (POI), and performing the task for each of the divided field work spaces. It includes the step of giving.

In addition, the augmented reality content-based work management system and method according to an embodiment of the present invention comprises the steps of setting a part of the at least one or more field work spaces as a safety management area, and the field worker entering the set safety management area In this case, the method further includes providing a predetermined alarm to the field device of the field worker.

In addition, the obtaining of the work data may include obtaining work environment setting data, which is information for setting at least one of whether to display the augmented reality content, a display position, a display method, and whether to provide legacy content; and setting the augmented reality content to be augmented and displayed on the field device based on the obtained work environment setting data.

In addition, in the augmented reality content-based work management system and method according to an embodiment of the present invention, the work environment setting data is at least one or more generated by dividing the field device or the space in which the field work is performed into at least one area. It is set for each on-site work space.

Augmented reality content-based work management system and method according to an embodiment of the present invention, by implementing a remote collaboration service based on the augmented reality content, remotely based on the augmented reality content to perform on-site management and guidance (Guidance) to perform can have an effect.

In addition, the augmented reality content-based work management system and method according to an embodiment of the present invention, by implementing a workstation service capable of managing a user (ie, a worker) and the corresponding work remotely located at the work site, the overall industrial site It has the effect of being able to easily control the progress from a macro perspective.

In addition, the augmented reality content-based task management system and method according to an embodiment of the present invention implements collaborative communication between users (ie, managers and workers) located remotely from each other based on a virtual environment, thereby creating augmented reality content Through the realistic screen, it is possible to provide easy and clear work guidance to remote field workers, which has the effect of enabling workers to understand and perform work more intuitively.

In addition, the augmented reality content-based task management system and method according to an embodiment of the present invention generates or updates augmented reality content based on real-time data generated through the execution of a task, thereby providing remote collaborative communication in real time At the same time, it has the effect of improving the safety of work by improving the accuracy of the information provided to workers at the industrial site while improving the quality of the work.

However, the effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood from the description below.

1 is a conceptual diagram of an augmented reality content-based task management system according to an embodiment of the present invention.
2 is an example of a concept for explaining an installed server system in the physical space of the site according to an embodiment of the present invention.
3 is an example of a concept for explaining an on-site twin model according to an embodiment of the present invention.
4 shows an example of experiencing an augmented reality environment through a wearable type computing device according to an embodiment of the present invention.
5 is an example of an internal block diagram of a workstation device according to an embodiment of the present invention.
6 is a flowchart illustrating an augmented reality content-based task management method according to an embodiment of the present invention.
7 is an example showing a workstation service environment according to an embodiment of the present invention.
8 is an example of a diagram for explaining a method of configuring a workspace according to an embodiment of the present invention.
9 is an example of a drawing for explaining a method of assigning a task for each field worker according to an embodiment of the present invention.
10 is an example of input data based on augmented reality in the field device according to an embodiment of the present invention.
11 and 12 are examples of a state showing process control content according to an embodiment of the present invention.
13 is an example of a state of showing feedback management content according to an embodiment of the present invention.
14 is an example of generating sensing content based on sensing data according to an embodiment of the present invention.
15 is a flowchart illustrating a method of performing real-time remote collaborative communication based on augmented reality content according to an embodiment of the present invention.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms. In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense. Also, the singular expression includes the plural expression unless the context clearly dictates otherwise. In addition, terms such as include or have means that the features or components described in the specification are present, and do not preclude the possibility that one or more other features or components will be added. In addition, in the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

<Detailed composition of augmented reality content-based task management system>

1 is a conceptual diagram of an augmented reality content-based task management system according to an embodiment of the present invention.

Referring to FIG. 1 , the augmented reality content-based work management system according to an embodiment of the present invention includes a field device 100: 101, 102. 103, a workstation device 300, a field sensor system 400, and a workstation. It may include a server system 500 .

Each of these components of FIG. 1 may be connected through a network. The network refers to a connection structure capable of exchanging information between respective nodes such as the field device 100, the workstation device 300, the field sensor system 400, and the workstation server system 500, and such a network Examples of 3GPP (3rd Generation Partnership Project) network, LTE (Long Term Evolution) network, WIMAX (World Interoperability for Microwave Access) network, Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network) ), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, UWB (Ultra Wide Band) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. various wired and wireless networks may include, but is not limited thereto.

<Augmented reality content-based task management system>

In an embodiment of the present invention, the augmented reality content-based work management system enables a remote user (ie, a remote manager) to collectively manage a user (ie, a field worker) located at a job site and tasks performed at the job site A workstation service may be provided.

Here, the remote manager means a person who monitors the on-site situation at a location remote from the site where the work is being performed, and provides work guidance to field workers based on augmented reality communication as virtual content. do.

In addition, the field worker means a person who performs work by receiving information related to work based on the augmented reality environment provided through the work guidance received at the work site.

In detail, the workstation service according to the embodiment may include various services required for field work and overall management of workers.

In an embodiment, the workstation service may include a customized process management service, an augmented reality content link service, a maintenance management service, an augmented reality-based remote collaboration service, and/or a feed-based process management service.

Here, the customized process management service may be a service that provides a function that can be used by setting a process progress status management platform for on-site work and workers according to user (master) definition.

In detail, in the embodiment, the augmented reality content-based work management system may set process steps necessary for performing tasks for each field work.

For example, the augmented reality content-based work management system may set the process step for the first task by dividing it into a first sub-task, a second sub-task, and a third sub-task step.

In addition, in an embodiment, the augmented reality content-based work management system may set a name for each process step.

In addition, the augmented reality content-based work management system may designate a workspace and/or field worker to perform a corresponding task for each set process step.

In addition, in an embodiment, the augmented reality content-based work management system may collect work data according to work performance for each process step, and visualize and provide process progress based on the collected work data.

In an embodiment, the augmented reality content-based task management system can convert and analyze task data according to task performance into a process progress rate, and visualize task progress information as a graph (eg, table-based visualization according to user settings or Automation graph-based visualization, etc.) can be provided.

In addition, the augmented reality content link service refers to a tag (eg, a QR code and / or barcode, etc.) may be a service provided through a link to the field device 100 that satisfies).

In addition, the augmented reality content link service generates and stores augmented reality content in advance without additional user designation, and if there is an abnormal facility within a predetermined area around the user (ie, field worker) at the work site, it matches the abnormal facility It may be a service that detects the augmented reality content and provides it with an alarm to the field device 100 of the corresponding user.

In detail, in an embodiment, the augmented reality content-based task management system may pre-create augmented reality content without a designated user and upload it to the workstation service.

At this time, the augmented reality content-based task management system may connect and upload tag information such as location information (GPS, NFC, RFID, Bluetooth and/or UWB, etc.) and/or equipment information matching the corresponding augmented reality content. have.

In addition, the augmented reality content-based work management system may determine whether abnormal facilities exist within a predetermined area around the user (ie, field worker) of the work site.

And the augmented reality content-based task management system, when it is determined that an abnormal facility exists within the corresponding area, detects augmented reality content matching the corresponding abnormal facility and provides it to the user's field device 100 along with an alarm. .

That is, the augmented reality content-based work management system may provide the corresponding augmented reality content with an alarm to the field device 100 of a field worker entering a predetermined area around the abnormal facility.

Through this, the augmented reality content-based work management system can effectively assist in the progress of maintenance work for abnormal equipment generated at the work site.

In addition, the augmented reality content-based task management system may perform task data confirmation and augmented reality content update based on the maintenance of abnormal facilities based on the provided augmented reality content.

In addition, the maintenance management service may be a service providing a process for managing a maintenance history for a work target object based on augmented reality content.

The maintenance management service may generate and provide an immersive electronic document for the maintenance history of the work target object, and may manage the maintenance history based on a user input for the provided immersive electronic document.

In detail, the augmented reality content-based work management system may generate an immersive electronic document implemented in the form of a process checklist and/or questionnaire response sheet related to maintenance work.

In addition, the augmented reality content-based task management system may set a target for the generated immersive electronic document (ie, a user (field worker) who will perform a questionnaire response using the corresponding electronic document).

For example, the augmented reality content-based work management system may set the target as a specific field worker, a specific workspace member, or all users.

In addition, in the augmented reality content-based work management system, when the field device 100 of the set target detects a work target object related to the corresponding maintenance work, it can provide augmented realistic electronic documents related to the corresponding maintenance work. .

In addition, the augmented reality content-based task management system may obtain a user input for the augmented immersive electronic document from the subject's field device 100 .

And the augmented reality content-based task management system may generate and share a response report based on the acquired user input.

In addition, the augmented reality content-based work management system may generate and share visualization data such as a maintenance history management graph, an analysis graph, and/or a dashboard based on the generated response report.

In addition, the augmented reality-based remote collaboration service may be a service capable of sharing work guidance by performing communication based on augmented reality content between remote users.

Here, the work guidance according to the embodiment is information that provides data necessary or assisting in performing the work, and the work manual content, which is content that visualizes the work order and/or method, etc. in the embodiment, and comments related to the work It may include chat content that provides and/or attachment content for sharing files (such as documents) related to tasks.

In detail, in an embodiment, the augmented reality content-based work management system may transmit pre-produced augmented reality content to field workers.

In this case, the field device 100 of the field worker who has received the augmented reality content may view the received augmented reality content based on the field image being photographed in real time.

Here, the augmented reality content-based work management system and the field device 100 may mutually share the augmented reality content augmented with the field image and the field image.

Based on this, the augmented reality content-based task management system and the field device 100 may perform remote collaborative communication based on augmented reality content.

In an embodiment, the augmented reality content-based work management system and the field device 100 may perform remote collaborative communication while sharing work guidance, chatting text and/or attachments, etc., based on the augmented reality content.

In addition, the augmented reality-based remote collaboration service may further include a feed-based process management service.

Here, the feed-based process management service may be a service capable of generating and sharing feedback based on augmented reality content matching the corresponding feedback request information based on the feedback request information generated from the field device 100 of the field worker. have.

In detail, in the embodiment, the augmented reality content-based work management system includes a field image recorded from the field device 100 of a field worker and/or information that the field worker requests for advice on a specific job (eg, inquiry text, etc.) ), including feedback request information, can be obtained.

In addition, the augmented reality content-based task management system may generate feedback providing information that provides feedback on the acquired feedback request information.

For example, the augmented reality content-based task management system may generate feedback providing information including guidance text for feedback request information, an attached file and/or a related manual, and the like.

In addition, in an embodiment, the augmented reality content-based work management system may provide the generated feedback providing information to the field device 100 of the corresponding field worker to promote remote collaborative communication.

On the other hand, the augmented reality content-based work management system can accurately monitor the work status of field workers in real time through the on-site twin model corresponding to the physical space of the work site in real time and the camera system placed on the site.

Here, the on-site twin model may be a three-dimensional virtual model generated by mapping a physical space to a three-dimensional virtual space based on data (eg, image and depth information) scanned in the physical space of the site. Therefore, the 3D space coordinates of the real site and the virtual coordinates of the 3D virtual space can be matched, and the real object of the 3D space is a virtual object with the virtual coordinates of the 3D virtual space matched with the real coordinates of the real object. It can be included in the field twin model.

In particular, the augmented reality content-based work management system provides a virtual camera image that displays information about the camera system and can remotely control the camera system on the site twin model, so that the remote manager shoots the desired work site. can be printed out.

In addition, the augmented reality content-based work management system detects and outputs a field image captured by the real object after presetting a work area for the real object/real object, which is the work target, so that the remote manager can work on the real object can be provided for monitoring.

In addition, the augmented reality content-based work management system may provide an interface for generating work guidance corresponding to a physical space other than the field of view of the field worker through the field image of the field twin model or the camera system.

- Workstation Server System (500: Workstation Server Systems)

In an embodiment of the present invention, the workstation server system 500 (hereinafter, server system) may provide a workstation service capable of remotely managing on-site work and workers based on augmented reality content.

In detail, the server system 500 may manage various business data about a process progress state and/or a worker in charge related to on-site work managed on the workstation service.

Also, in the embodiment, the server system 500 may manage maintenance-related data for a work target object in the field.

In addition, the server system 500 may store and manage augmented reality content pre-produced in relation to the work target object.

In addition, the server system 500, based on the work data obtained from the field device 100 and/or the sensing data obtained from the field sensor system 400, etc., may generate and manage augmented reality content related to the work site. have.

In addition, the server system 500 models a virtual three-dimensional site twin model based on the three-dimensional scan data of the physical space of the work site, and provides a real object or site in the physical space of the site to the modeled site twin model. A process that reflects information about workers in real time can be performed.

At this time, at least some of the above processes may be performed in the server system 500 and the rest may be performed in the workstation device 300 of the remote administrator.

In addition, the server system 500 may relay augmented reality communication data exchange between the workstation device 300 of the remote manager and the field device 100 to provide mutual augmented reality communication.

Also, the server system 500 may update and provide the virtual content in real time based on real-time data generated during the progress of the job.

In an embodiment, the server system 500 as above includes a work management server 510 , a maintenance management server 520 , a virtual content server 530 , a monitoring server 540 , and a field space database server 550 . can do.

In detail, the work management server 510 may manage various work data for a process progress state and/or a worker in charge related to on-site work managed on the workstation service.

In an embodiment, the task management server 510 may manage process progress data for a plurality of field jobs and workers.

Also, the work management server 510 may provide work information based on augmented reality content for each field device 100 of a plurality of field workers.

Here, the task information according to the embodiment is basic information that can explain and assist the work to be performed by field workers, and is related to worker designation information, work target object (target) designation information, work content information and/or work. It may include necessary augmented reality content information and the like.

In addition, the maintenance management server 520 may manage maintenance-related data for a work target object in the field.

In an embodiment, the maintenance management server 520 may generate a realistic electronic document implemented in the form of a process checklist and/or questionnaire response sheet related to maintenance work.

Also, the maintenance management server 520 may acquire job data generated by a field worker from the field device 100 based on the generated immersive electronic document.

In addition, the maintenance management server 520 may create and manage management content based on the obtained job data.

Here, the management content according to the embodiment may be work management content generated based on work data obtained from field workers.

In an embodiment, the management content may include process management content, maintenance management content, abnormal equipment management content, and/or feedback management content. A detailed description thereof will be provided later.

In addition, the virtual content server 530 generates and manages augmented reality content related to the work site based on the work data obtained from the field device 100 and/or the sensing data obtained from the field sensor system 400 . can

In an embodiment, the augmented reality content may be content that is 3D augmented in a virtual environment and provides predetermined information to a user.

Such augmented reality content may include work guidance content, maintenance management content, abnormal facility management content, sensing content and/or feedback management content, and the like. A detailed description thereof will be provided later.

In addition, the virtual content server 530 may store and manage augmented reality content pre-produced in relation to the work target object.

In addition, the virtual content server 530, the previously produced augmented reality content, a predetermined condition (for example, a specific field device 100 is a tag (eg, QR code and / or barcode) arranged in a predetermined position etc.) may be provided through a link to the field device 100 that satisfies).

In addition, the virtual content server 530 may generate a site twin model corresponding to the physical space of the site in a virtual 3D space based on data obtained by 3D scanning the physical space of the site.

For example, the virtual content server 530 may physically create a 3D virtual space based on an image of a physical space of the field and spatial information (eg, depth data) scanned on the positions of real objects located in the physical space. It is possible to create an on-site twin model that models virtual objects corresponding to real objects in space.

Also, the virtual content server 530 may generate an on-site twin model while reflecting the remote administrator's input from the remote administrator's workstation device 300 .

Also, the monitoring server 540 may monitor and manage overall functional operations performed in the workstation service.

In an embodiment, the monitoring server 540 may check the real-time on-site work and process progress for the operator, and check the real-time on-site remote collaboration situation.

In addition, the monitoring server 540 may receive information about real objects and field workers in the physical space from the field sensor system 400 and/or the computing device of field workers, and update the field twin model in real time. .

For example, the monitoring server 540 receives the sensor data measured by the real object from the on-site sensor system 400 in real time, converts the received sensor data into virtual content, and then around the virtual object image of the on-site twin model. By creating a site twin model in which the virtual content is inserted in the , a remote manager can effectively monitor the site situation through the created site twin model.

In addition, the monitoring server 540 receives, in real time, the location information and the on-site image of at least one field worker from the field sensor system 400, and a virtual on-site twin model corresponding to the received real-time location information of the field worker. By creating a field twin model in which virtual content representing the field worker is inserted in the location, it is possible to provide a remote manager to identify the location of the field worker through the generated field twin model.

In addition, the monitoring server 540 records and stores the work history according to the work of the field worker in the virtual object corresponding to the field twin model and/or the virtual content matched to the field worker, thereby working on the field worker and the real object History can be monitored.

In addition, the monitoring server 540 may detect from the field sensor system 400 that the field worker enters a preset dangerous area.

At this time, the monitoring server 540 may provide a notification based on the augmented reality environment to the field worker computing device, and the remote manager computing device (ie, the workstation device 300 ) may have on-site twin model, augmented reality communication or / And it can provide field worker risk monitoring according to the augmented reality environment.

In addition, the monitoring server 540 may include a communication facility for data relay, a computing device, and the like, and relay communication data to and from the computing devices through a wired/wireless communication network to transmit/receive communication data between computing devices, thereby performing augmented reality communication between computing devices. environment can be provided.

For example, the monitoring server 540 may serve to relay communication data transmitted/received between the remote manager computing device (workstation device 300 ) and the field worker computing device.

In addition, the site space database server 550 may store and manage various application programs, applications, commands and/or data for implementing the workstation service.

In an embodiment, the site space database server 550 may store augmented reality content data for implementing an augmented reality environment.

The on-site spatial database server 550 may match the augmented reality content to a real object (eg, a marker) or spatial coordinates to store the augmented reality content.

In addition, the site space database server 550 may match and store the augmented reality content with the virtual coordinates of the three-dimensional virtual space of the site twin model.

In addition, the site space database server 550 may serve as an augmented reality content source for delivering augmented reality content matched to the surrounding physical space of the computing device upon request of the computing device.

In addition, the site space database server 550 may further include a three-dimensional virtual image modeling a virtual object for constructing a field twin model, and augmented reality content obtained by informatizing field sensing data for the virtual object.

In addition, the site space database server 550 may scan the site physical space or perform 3D space modeling to store information data on the physical space.

In addition, the site space database server 550 may further store characteristic information obtained by image-learning a real object, a marker, etc. in a physical space, and the characteristic information may be matched with the spatial information and stored.

That is, the server system 500 according to the embodiment transmits augmented reality content data and spatial information data for the surrounding physical space of the computing device together, and provides an augmented reality environment through the computing device and/or on-site twin model. can do.

The server system 500 may include at least one computing server, a computing device, and a database server, and may include processors for data processing and memories for storing instructions for providing a communication relay service.

- Site Sensor Systems (400: Site Sensor Systems)

The field sensor system 400 may include various sensors that detect and measure various information about the physical space of the field (eg, information of the environment, machinery and/or field workers).

For example. The field sensor system 400 may include at least one of a temperature sensor, a humidity sensor, a pressure sensor, a flow sensor, a magnetic sensor, an optical sensor, an acoustic sensor, an image sensor, a current sensor, and a gas sensor disposed in the field. have.

The on-site sensor system 400 may be disposed at a predetermined location (eg, a location where it is easy to sense a work target and/or a field worker, etc.) of the work site, or be implemented by being included in the work target object itself. can

In an embodiment, the field sensor system 400 includes a positioning sensor system 410 , a camera system 420 , a device sensor system 430 , and a gateway 440 .

The positioning sensor system 410 may sense the location information of the field worker in real time.

The positioning sensor system 410 is an optoelectronic sensor, an ultrasonic sensor, a gate type sensor, a thermal sensor, a laser sensor, a lidar sensor, a Wi-Fi sensor, a beacon sensor, a UWB (Ultra wide band) sensor, Bluetooth It is possible to detect the location information of field workers based on technologies such as sensors, image analysis sensors, RFID sensors, PDRs, and GPS.

In an embodiment, the positioning sensor system 410 may be a high-precision UWB-based real-time positioning sensor system 410 to precisely monitor a work site or detect a detailed movement of a field worker located in a hazardous area.

In detail, the positioning sensor system 410 may include a plurality of anchors (UWB Anchor) installed in a physical space for receiving a radio signal generated from a tag (UWB Tag) possessed by a field worker.

The tag of the field worker may be provided by the field user in the form of a separate wireless signal transmitting device, but in the embodiment, it will be described as included in the field device 100 .

Alternatively, according to an embodiment, the positioning sensor system 410 may be an RFID (Radio-Frequency Identification)-based real-time positioning sensor system 410 capable of detecting a remote field worker and/or an object at a work site.

In this case, the positioning sensor system 410 according to the embodiment may perform electromagnetic induction type communication for recognizing far-distance information using a frequency (radio wave).

The radio signal generated from the tag may be a UWB radio signal that transmits the identification information (ID, identification) and time stamp of the on-site user in nanoseconds or milliseconds.

Since the UWB-based positioning technique has been standardized for commercial use in IEEE 802.15.3a and IEEE 802.15.4a, it will be described as using the standardized communication technique.

In detail, IR-UWB (Impulse-Radio Ultra Wide Band) can transmit data at a high speed of over 100Mpbs in the 3.1~10.6 GKHZ band depending on the transmission method, and has the advantage of enabling high-precision location tracking within tens of centimeters with low power. .

A plurality of anchors arranged at a predetermined interval in a physical space receive a radio signal of the tag and transmit positioning sensor data to the server system 500 to determine the position of the tag (eg, the position of a field worker) or The tag position can be calculated directly.

In an embodiment, the server system 500 calculates a TDoA (Time Difference of Arrival) method or RTT (Round Trip Time) for calculating the location of a tag based on the arrival time of a radio signal of the tag received from time-synchronized anchors And through the positioning sensor data received from the positioning sensor system 410 through at least one method of the TWR (Two-Way Ranging) method of converting the calculated RTT into a distance, real-time location information (field worker identification information) and position) can be precisely calculated.

The positioning sensor system 410 may have an anchor arrangement structure so that a shadow area of an anchor is not generated when an object of a predetermined size or more is located in a one layer (eg, one layer) of a predetermined height in a physical space.

In addition, the on-site sensor system 400 may include a camera system 420 having a plurality of cameras that acquire a field image by photographing a region of the on-site physical space.

The camera system 420 includes at least one remote control camera (eg, PTZ camera) capable of changing a shooting area according to an omnidirectional camera (eg, a 360-degree camera) and/or a remote manager that shoots omnidirectionally from an installed position. may include

The camera system 420 may include an omnidirectional camera disposed at a predetermined interval in the field, and the omnidirectional camera may acquire an omnidirectional field image by photographing 360 degrees at the placement position.

In addition, the camera system 420 may include a remote control camera that can remotely control a shooting direction and enlargement/reduction in the field. The remote control camera may acquire a field image for a shooting direction and an enlarged/reduced shooting area automatically or under the control of a remote manager.

The device sensor system 430 may include sensors for detecting various information of devices disposed in the field.

In detail, the device sensor system 430 may acquire device-related information, including an IOT sensor disposed in the device, and sensors for sensing the device and the device surrounding environment.

The sensor data acquired through the on-site sensor system 400 may be transmitted to the gateway 440 including at least one or more access points.

The gateway 440 may collect sensor data obtained from the on-site sensor system 400 , and may serve to transmit the collected sensor data to the server system 500 .

To this end, the gateway 440 may exchange data with the on-site sensor systems 400 through various wired/wireless communication methods.

- Computing Device

In an embodiment, the computing device includes a computing device (100: hereinafter, “field device”) of a field worker located in the field, and a computing device of a remote administrator who remotely monitors the on-site situation (hereinafter, “workstation device”) (300: hereinafter, “workstation device”). can be divided into

Here, the workstation device 300 according to the embodiment may execute a workstation service platform by accessing a workstation server based on a network, and may perform a workstation service that remotely manages the field and workers through this. have.

In addition, the workstation device 300 may remotely monitor the site through the site twin model reflecting the site physical space according to the control of the installed work management application.

Referring to FIG. 3 , for example, the workstation device 300 may generate and output a virtual site twin model 50 corresponding to the physical space 10 of an actual site.

In the on-site twin model 50, a virtual object image corresponding to a real object may be displayed, and augmented reality content generated based on sensor data measured from the real object is displayed around the virtual object image, so that the real object real-time monitoring can be performed.

In addition, the field twin model 50 may display real-time location information of field workers based on the sensor data of the field sensor system 400 as augmented reality contents (VR), and field workers in the augmented reality contents (VR) Job history, job information, etc. inputted by or confirmed through sensor data may be further included.

In addition, the workstation device 300 performs augmented reality communication with the field device 100 based on the field twin model 50, and provides work instructions or work guidance through augmented reality content matched to voice and real objects. can provide

In addition, the workstation device 300 may monitor that the field worker enters the danger zone through the field twin model 50 and communicate about the danger situation through augmented reality communication.

According to an embodiment, a remote manager located at the site may monitor the site through the workstation device 300 , but in the following description, the remote manager will be described based on being located outside the site.

The on-site device 100 performs an augmented reality-based work assistance interface in which a work guide and work history are input in an augmented reality environment through augmented reality content matched to a real object in a physical space according to the control of the installed work management application. can do.

In addition, the work management application of the field device 100 may perform augmented reality communication with the workstation device 300 to receive a work instruction or a work guide from a remote manager in an augmented reality environment.

In addition, the work management application of the field device 100 may provide a diversified notification of a dangerous situation when entering the danger zone through the augmented reality environment.

Such a computing device may include various types of computing devices (eg, wearable type, mobile type, or desktop type) in which the task management application and/or the task management application are installed.

In the following description, the field device 100 is described with reference to including the wearable type computing devices 101 and 102 and the mobile type computing device 103 , and the workstation device 300 is a desktop type computing device 300 . It is described as including

However, an embodiment in which the workstation device 300 includes a wearable type or a mobile type computing device may be possible.

1. Wearable type computing device (101, 102)

4 is a conceptual diagram of experiencing an augmented reality environment through wearable-type computing devices 101 and 102 according to an embodiment of the present invention.

The computing device according to the embodiment may include wearable type computing devices 101 and 102 such as smart glasses display or head mounted display (HMD).

The smart glass-type computing device 101 is a display system including glasses that display augmented reality content (eg, virtual object image) on the user's field of vision while transmitting light so that the user can see the surrounding physical space while worn. may include

In detail, the smart glass-type computing device 101 of the embodiment is a transparent glass that transmits light from the surrounding physical space to reach the user's eyes and at the same time reflects the augmented reality content displayed by the display system toward the user's eyes. It may include a display.

For example, referring to FIG. 4 , the task management application of the smart glass type computing device 101 may recognize an image of a real object (RO) and a marker (MK) within the real object in the surrounding physical space 10 . And, it is possible to control to display the augmented reality content (VC1) in the user's field of view corresponding to the recognized marker (MK).

In addition, the task management application may recognize the learned real object, and may control to display the augmented reality content VC2 in the user's field of view corresponding to the position of the recognized real object.

In addition, the task management application may recognize a physical space through a learned real object or marker (MK), and may display augmented reality content matched to a specific location in the recognized space to correspond to the user's field of view.

Such augmented reality content may include visual content such as an image or video that may be displayed in a portion of the user's field of view in the smart glass type computing device 101 .

For example, the augmented reality content may include virtual object images that overlay various parts of a physical space. This virtual object image may be rendered as a 2D image or a 3D image.

The head mounted display type computing device 102 may block light to the surrounding physical space to view the displayed image only by the display system. The head-mounted display type computing device may output a 3D image by outputting different images with an offset in parallax to each of the left and right eye displays in order to recognize the 3D scene.

In addition, the head-mounted display type computing device 102 may also provide an augmented reality environment by outputting an image of the surrounding physical space and augmented reality content generated based on the captured image as a 3D image.

Hereinafter, specific components will be described with a focus on the smart glass type computing device 101 among these wearable type computing devices 101 and 102 .

The smart glass type computing device 101 according to the exemplary implementation may include a memory including a task management application, a processor assembly, a communication module, an interface module, an input system, a sensor system, and a display system. In addition, the components may be implemented to be included in a housing of the computing device 101 .

In the memory , a task management application is stored, and the task management application may include augmented reality content for providing an augmented reality environment, an image buffer, a location engine, an augmented reality content display engine, and the like. That is, the memory may store commands and data that may be used to create an augmented reality environment.

In an embodiment, the task management application may include a communication application for performing communication based on the augmented reality environment. The communication application may include various applications, engines, data and commands for providing a network speed responsive augmented reality communication service.

In addition, the task management application may include a danger notification application that provides a notification of a dangerous situation when entering the danger zone based on the augmented reality environment.

In addition, the memory may include at least one or more non-transitory computer-readable storage media and a temporary computer-readable storage medium. For example, the memory may be various storage devices such as ROM, EPROM, flash drive, hard drive, and the like, and may include web storage that performs a storage function of the memory on the Internet.

The processor assembly may include at least one processor capable of executing instructions of a task management application stored in a memory to perform various tasks for creating an augmented reality environment.

In an embodiment, the processor assembly may control the overall operation of the components through the task management application in the memory to provide the augmented reality work guide, augmented reality communication, and risk notification service.

For example, the processor assembly may recognize a real object from an image obtained based on an image sensor, and generate and display an augmented reality image matching augmented reality content to the recognized real object. 101) can be controlled.

Such a processor assembly may include a central processing unit (CPU) and/or a graphics processor unit (GPU). In addition, the processor assembly, ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers (controllers), micro It may be implemented by including at least one of controllers (micro-controllers), microprocessors, and other electrical units for performing functions.

The communication module may include one or more devices for communicating with other computing devices (eg, server system 500 ). Such a communication module may communicate through a wireless network.

In detail, the communication module may communicate with a computing device storing an augmented reality content source for implementing an augmented reality environment, and may communicate with various user input components such as a controller that has received a user input.

In an embodiment, the communication module may transmit and receive communication data related to the network speed responsive augmented reality communication service with the server system 500 and/or other computing devices.

Such a communication module, the technical standards or communication methods for mobile communication (eg, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), 5G NR (New Radio), WIFI) or short-range communication Data may be wirelessly transmitted/received with at least one of a base station, an external terminal, and an arbitrary server on a mobile communication network constructed through a communication device capable of performing the above method.

The sensor system may include various sensors such as an image sensor, a position sensor (IMU), an audio sensor, a distance sensor, a proximity sensor, a contact sensor, or a tag.

The tag can transmit a wireless signal including identification information and real-time time information of a field worker at regular intervals.

The tag may be owned by a field user as a separate wireless signal transmitting device, but in an embodiment, it will be described as being included in the field worker's computing device.

In detail, the tag may be a UWB radio signal that transmits the identification information (ID, identification) and time stamp of the on-site user in nanoseconds or milliseconds.

Since the UWB-based positioning technique has been standardized for commercial use in IEEE 802.15.3a and IEEE 802.15.4a, it will be described as using the standardized communication technique.

In detail, IR-UWB (Impulse-Radio Ultra Wide Band) can transmit data at a high speed of over 100Mpbs in the 3.1~10.6 GKHZ band depending on the transmission method, and has the advantage of enabling high-precision location tracking within tens of centimeters with low power. .

The image sensor may capture an image and/or an image of the physical space 10 around the smart glass type computing device 101 .

In an embodiment, the image sensor may be obtained by photographing an image related to a network speed responsive augmented reality communication service.

In addition, the image sensor is disposed on the front or / and rear of the smart glass type computing device 101 to obtain an image by photographing the disposed direction side, and is disposed toward the outside of the smart glass type computing device 101 . It is possible to photograph the physical space 10 such as a work site through the camera.

Such an image sensor may include an image sensor and an image processing module. Specifically, the image sensor may process a still image or a moving image obtained by an image sensor (eg, CMOS or CCD).

In addition, the image sensor may process a still image or a moving image obtained through the image sensor using the image processing module to extract necessary information, and transmit the extracted information to the processor.

Such an image sensor may be a camera assembly including at least one camera. The camera assembly may include a general camera that captures a visible light band, and may further include a special camera such as an infrared camera or a stereo camera.

The image sensor IMU may sense at least one of motion and acceleration of the smart glass type computing device 101 . For example, it may consist of a combination of various position sensors such as an accelerometer, a gyroscope, and a magnetometer. In addition, by interworking with a location communication module such as GPS of the communication module, spatial information about the physical space 10 around the smart glass type computing device 101 may be recognized.

In addition, the IMU 163 may detect information for detecting and tracking the user's gaze direction and head movement based on the detected position and direction.

Further, in some implementations, a task management application may use such an IMU 163 and an image sensor to determine the location and orientation of a user within the physical space 10 or to recognize a feature or object within the physical space 10 . .

The audio sensor may recognize a sound around the smart glass type computing device 101 .

In detail, the audio sensor may include a microphone capable of detecting a voice input of a user of the smart glass type computing device 101 .

In an embodiment, the audio sensor may receive voice data of communication data to be transmitted through the augmented reality communication service from the user.

The interface module may communicatively connect the smart glass type computing device 101 with one or more other devices. Specifically, the interface module may include wired and/or wireless communication devices that are compatible with one or more different communication protocols.

The smart glass type computing device 101 may be connected to various input/output devices through such an interface module.

For example, the interface module may be connected to an audio output device such as a headset port or a speaker to output audio.

Although it has been described that the audio output device is connected through the interface module by way of example, an embodiment in which the smart glass type computing device 101 is installed may be included.

Such an interface module, a wired / wireless headset port (port), an external charger port (port), a wired / wireless data port (port), a memory card (memory card) port, a port for connecting a device equipped with an identification module (port) ), at least one of an audio input/output (I/O) port, a video I/O (input/output) port, an earphone port, a power amplifier, an RF circuit, a transceiver and other communication circuits. It may be composed of

The input system may detect a user's input (eg, gesture, voice command, operation of a button, or other type of input) related to the network speed responsive augmented reality communication service.

Specifically, the input system may include a button, a touch sensor, and an image sensor for receiving user motion input.

Also, the input system may be connected to an external controller through an interface module to receive a user's input.

The display system is transparent to transmit the light from the physical space 10 around the smart glass type computing device 101 to reach the user's eyes, and at the same time reflect the augmented reality content displayed by the display system toward the user's eyes. It may include a glass display.

Such a display system may include a left display corresponding to the left eye of the user wearing the smart glass type computing device 101 and a right display corresponding to the right eye, wherein the left display and the right display are offset from each other by parallax. By outputting another image as augmented reality content, the user can recognize the augmented reality content as a three-dimensional image.

In an embodiment, the display system may output various information related to the network speed responsive augmented reality communication service as a graphic image.

Such displays include a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display. , a three-dimensional display (3D display), may include at least one of an electronic ink display (e-ink display).

These wearable-type computing devices 101 and 102 may be worn by field workers located in a physical space 10 such as a job site while performing field work.

2. Mobile type computing device (103)

In another example, the computing device may be a mobile device such as a smart phone or a tablet PC on which a task management application is installed.

The mobile type computing device 103 captures an image of the surrounding physical space 10 with an image sensor, and displays augmented reality content displayed by matching the captured image and the physical space 10 through a display system to augment A real environment may be provided to the user.

For example, the mobile type computing device 103 may include a smart phone, a mobile phone, a digital broadcasting terminal, personal digital assistants (PDA), a portable multimedia player (PMP), a tablet PC, and the like. have.

The task management application of the mobile type computing device 103 may capture an image of a real object RO and a marker MK in the surrounding physical space 10 and display the image by controlling the display system. In addition, the task management application may control to display the augmented reality content VC1 at a position corresponding to the recognized marker MK. In addition, the task management application may learn and recognize a specific real object, and may control to display the augmented reality content VC2 in the user's field of view corresponding to the recognized location of the specific real object.

The mobile type computing device 103 according to the example implementation may include a memory, a processor assembly, a communication module, an interface module, an input system, a sensor system, and a display system. These components may be configured to be included within the housing of the mobile type computing device 103 .

In the description of the components of the mobile type computing device 103, the overlapping contents will be replaced with the description of the components of the wearable type computing devices 101 and 102. Hereinafter, the wearable type computing devices 101 and 102 ) and the differences between them.

The components may be disposed within the housing of this mobile type computing device 103 , and the user interface may include a touch sensor on a display configured to receive user touch input.

In detail, the display system may include a display that outputs an image, and a touch sensor that detects a user's touch input.

Exemplarily, the display may be implemented as a touch screen by forming a layer structure with the touch sensor or being integrally formed therewith. Such a touch screen may function as a user input unit providing an input interface between the mobile type computing device 103 and the user, and may provide an output interface between the mobile type computing device 103 and the user.

Also, the sensor system includes an image sensor, and for example, the image sensor may be disposed on one side and the other side of the housing of the mobile type computing device 103 .

At this time, the image sensor on one side is oriented toward the physical space 10 to capture an image photographed in the physical space 10, and the image sensor on the other side is oriented toward the user side to control the user's view, gestures, etc. can be filmed

Such a mobile type computing device 103 may be suitable for a field worker located in a physical space 10 such as a job site.

3. Desktop type computing device (300)

5 is an example of an internal block diagram of a workstation device 300 according to an embodiment of the present invention.

In the description of the components of the desktop-type computing device 300, the overlapping contents will be replaced with the description of the components of the wearable-type computing devices 101 and 102. Hereinafter, the wearable type computing device 101, 102), focusing on the differences.

In such a desktop type computing device 300 , a work management application 311 for monitoring the site based on the site twin model 50 may be installed.

Referring to FIG. 5 , the task management application 311 of the desktop type computing device 300 may control the display device 370 to output a virtual field twin model 50 corresponding to a physical space, Augmented reality content (VR) may be displayed in the on-site twin model.

In the on-site twin model 50, a virtual object image corresponding to a real object may be displayed, and augmented reality content generated based on sensor data measured from the real object is displayed around the virtual object image, so that the real object real-time monitoring can be performed.

In addition, the task management application 311 of the desktop type computing device 300 is augmented reality content corresponding to the physical space of the field based on the on-site twin model 50, the on-site image or/and the photographed image of the field worker. An interface may be provided, and the augmented reality content generated in this way may be generated and displayed at a location corresponding to a physical space.

In addition, illustratively, the computing device 300 is based on wired/wireless communication, such as a fixed desktop PC, a laptop computer, and a personal computer such as an ultrabook, in which the task management application 311 is installed. It may further include a device installed with a program for executing a network speed responsive augmented reality communication service.

This desktop-type computing device 300 receives an image of the surrounding physical space from another user's computing device 300, and augments and displays the received image and augmented reality content matched to the physical space, An augmented reality environment may be provided to the user.

In addition, the desktop type computing device 300 includes the user interface system 350 to receive user input (eg, touch input, mouse input, keyboard input, gesture input, motion input using a guide tool, etc.). can

Illustratively, the computing device 300 includes a mouse 351 , a keyboard 352 , a gesture input controller, an image sensor 361 (eg, a camera) and an audio sensor (eg, a camera) using the user interface system 350 with various communication protocols. 365), etc., may be connected to at least one device to obtain a user input.

In addition, the desktop type computing device 300 may be connected to an external output device through the user interface system 350 , for example, the display device 370 , an audio output device, or the like.

In an exemplary implementation, the task management application 311 of the desktop computing device 300 may obtain and output an image of the computing device 300 of another user through the display device 370 , and the image or the field A user input corresponding to the twin model 50 may be received, and an image or augmented reality content corresponding to the on-site twin model 50 may be generated according to the received user input.

In addition, the task management application 311 may receive data from a sensor system in the physical space of the image or a previously matched augmented reality content source, and generate the received data as augmented reality content matched with the image.

In addition, the task management application 311 may provide an augmented reality environment to the user by overlaying and outputting the augmented reality content generated in this way on the captured image, on-site image or on-site twin model 50 output from the display device. have.

In addition, the task management application 311 transmits the augmented reality content generated through the communication module 330 as communication data, and communicates the augmented reality content corresponding to the physical space 10 along with voice and image for communication. It can be used as a medium.

The desktop type computing device 300 according to the example implementation may include a memory 310 , a processor assembly 320 , a communication module 330 , a user interface system 350 , and an input system 340 . These components may be configured to be included within a housing of the computing device 300 .

In the description of the above components of the desktop type computing device 300 , the overlapping contents will be replaced with the description of the components of the wearable type computing device 300 .

The desktop-type computing device 300 may be advantageously used by a remote manager who remotely transmits instructions or necessary information by interworking with the computing device 300 of a field worker.

In the following embodiment, the workstation device 300 , which is a computing device of a remote manager who remotely monitors the on-site situation, will be described with reference to the desktop type computing device 300 , but is not limited thereto.

- Augmented reality content-based task management method

Hereinafter, a method in which the job management application 311 of the workstation device 300 manages field work based on augmented reality content will be described in detail with reference to the accompanying drawings.

However, according to an embodiment, the mobile type computing device 103 and/or the wearable type computing devices 101 and 102 may also have a job management application installed to perform on-site monitoring and job management in real time.

6 is a flowchart illustrating an augmented reality content-based task management method according to an embodiment of the present invention.

Referring to FIG. 6 , in the embodiment, the work management application 311 of the workstation device 300 may build a workstation and create a workspace. (S101)

Here, the workstation according to the embodiment may mean a web or mobile service environment in which an augmented reality content-based work management service (workstation service) is provided.

In this case, in the embodiment, the augmented reality content may be content that is three-dimensionally augmented in a virtual environment to provide predetermined information to the user.

In an embodiment, the augmented reality content includes work guidance content related to field work (in an embodiment, it may be established on a workstation service), process management content, maintenance management content, abnormal equipment management content, sensing content and/or feedback management content, and the like.

7 is an example showing a workstation service environment according to an embodiment of the present invention.

Turning again to FIG. 7 , in a detailed embodiment a workstation is configured to perform a workstation service by way of a console comprising all devices and/or equipment for performing a specific task related to the shop floor. It may be a service environment.

In addition, the workspace according to the embodiment may mean a work organization unit organized by a user by inviting a person concerned with his/her work in order to use a workstation service.

In detail, in the embodiment, users of the workstation service may be classified into workspace masters and/or members according to the user's authority.

Here, the workspace master user may be the authority of the workspace who owns the workspace, purchases a plan, and allocates it to members (members, etc.).

In addition, the workspace member user may mean a user who participated in the workspace by inviting the master user to the workspace.

In the following embodiment, a remote manager who monitors the site situation at a location remote from the site where the work is being performed and instructs or assists the work based on augmented reality communication to the field worker is described as a master user, and the A field worker who receives information related to work based on the augmented reality environment provided through the work guidance and performs work is described as a member user, but is not limited thereto.

In addition, in the embodiment, the plan may be information that can set the user's right to use the workspace service and/or the service availability period.

Here, the right to use the workspace service includes the Make right to create augmented reality content, the View right to implement visualization, and/or the Remote right to perform remote collaboration function operations. may include

8 is an example of a diagram for explaining a method of configuring a workspace according to an embodiment of the present invention.

Referring to FIG. 8 , in the embodiment, the job management application 311 of the workstation device 300 of the remote administrator (master user) may build a workstation providing a workstation service environment.

In addition, the work management application 311 may configure at least one workspace by inviting at least one or more work related persons as member users based on the built workstation.

In an embodiment, the job management application 311 interworks with at least one or more on-site devices 100 to input a user (ie, a remote administrator) and/or a predetermined criterion (eg, a preset job characteristic for a workspace and /or at least one user (ie, field workers) can be set as members of the workspace based on difficulty, etc.).

In addition, in an embodiment, the work management application 311 may allocate a plan for each invited at least one member user based on the workstation.

For example, the work management application 311 assigns a plan including a make right to a first member user invited to a workspace, assigns a plan including a view right to a second member user, and a third member Users can be assigned a plan that includes make rights, view rights, and remote rights.

In this case, in an embodiment, the field worker (member user) may participate in at least one workspace by receiving an invitation from at least one or more master users.

In addition, the field worker (member user) may select one of at least one or more workspaces to which he or she belongs, and may access a workstation of the selected workspace to use a workstation service.

In this way, the work management application 311 implements a location-independent connection between work-related persons by establishing a workstation and a workspace, thereby providing a customized process management service, augmented reality content link service, maintenance management service, and augmented reality-based It is possible to provide a workstation service that can remotely and collectively perform services necessary for field work and overall management of workers, such as a remote collaboration service and/or a feed-based process control service.

In addition, through this, the work management application 311 can remotely provide field management and guidance based on augmented reality content, and can easily manage the overall progress of the industrial site from a macroscopic point of view.

In addition, in an embodiment, the work management application 311 may grant tasks based on augmented reality content for each field device 100 (field workers). (S103)

9 is an example of a drawing for explaining a method of assigning a task for each field worker according to an embodiment of the present invention.

In detail, referring to FIG. 9 , in the embodiment, the job management application 311 provides job information indicating information on the job (job) assigned to each worker by interworking with the field device 100 for each of a plurality of field workers. can do.

Here, the task information according to the embodiment is basic information that can explain and assist the work to be performed by the field worker, and the worker designation information, the work target object (target) designation information, the work content information, and the need related to the work are reinforced It may include real content information and/or granted plan information, and the like.

At this time, in the embodiment, the task management application 311 may provide the necessary augmented reality content related to the task obtained based on the task target (target) designation information.

For example, when the work management application 311 is work information using the first object as a work target object (eg, a specific facility or device, etc.), the necessary augmented reality content information of the work information is related to the first object. Augmented reality content can be configured and provided.

In this way, the work management application 311 gives a task for each field device 100 (field worker), and by providing the augmented reality content necessary for the corresponding task, remote through a realistic screen through the augmented reality content It is possible to provide easy and clear work guidance to field workers in the

In another embodiment, the work management application 311 may divide the work site space, that is, the workstation service space, into at least one or more areas to generate at least one point of interest (POI).

Here, the field work space (POI) according to the embodiment may be a predetermined area set to classify the work site for each area and provide work information in units of the divided area.

In detail, the work management application 311 may provide work information for each field work space (POI) based on at least one generated field work space (POI).

At this time, the work management application 311 is a plan for augmented reality content linked to the corresponding field work space (POI) (that is, production, viewing and / or whether collaboration permission) can be set.

In addition, the work management application 311, when assigning work information for each field work space (POI), whether legacy data (Legacy data) interlocking in the field work space (POI) and / or UWB, RFID, beacon and / Alternatively, it is also possible to set whether to designate a separate work location based on other positioning methods such as Bluetooth.

In addition, the work management application 311 that has generated at least one field work space (POI) as described above is a safety management area that requires more attention when performing work on a part of the generated at least one field work space (POI). It can be set as a zone.

In detail, the job management application 311 is based on the actual location information of the job site (eg, location information located based on UWB-Anchor and/or RFID-Tag, etc.), corresponding to the actual location information. You can set up a field work space (POI).

In addition, the work management application 311 may set at least a portion of the set field work space (POI) as a safety management zone based on a predetermined criterion (eg, an area in which the number of maintenance is greater than or equal to a predetermined value).

At this time, when an arbitrary field worker enters the field work space (POI) set as the safety management area, the work management application 311 interworks with the field device 100 of the field worker to transmit an alarm for entering the safety management area. year can provide

In addition, the work management application 311 for performing assignment based on the field work space (POI), when an arbitrary field worker enters the field work space (POI), the work information assigned to the field work space (POI) (In the embodiment, it is possible to provide work content information, necessary augmented reality content information, grant plan information and/or work target object (target) designation information, etc.) .

In detail, the work management application 311, when an arbitrary field worker enters the field work space (POI) to which the predetermined work information is assigned, the work information assigned to the field work space (POI) of the field worker It can be transmitted to the field device 100 .

In addition, the work management application 311 transmits work information to the field device 100 of the field worker, so that the augmented reality content related to the field work space (POI) is augmented and displayed in the field device 100 can do.

In addition, in an embodiment, the task management application 311 may limit the right to use the augmented reality content provided through the work information, based on the plan assigned to each user.

For example, the work management application 311, when the plan assigned to the first member user includes the make right, the first member user having the make right edits and/or the augmented reality content of the assigned work information may be permitted to manufacture.

In this way, the task management application 311 may strengthen the security of data and/or information shared on the workstation service by limiting the usage right for the augmented reality content based on the plan allocated to each user.

In addition, in an embodiment, the work management application 311 may obtain work data from the field device 100 to generate management content. (S105)

Here, the job data according to the embodiment may mean predetermined job environment setting data and/or job performance result data obtained as the corresponding job is performed during or after completion of the on-site job.

Here, the work environment setting data according to the embodiment may be information that sets whether to display augmented reality content provided when performing field work, a display location, a display method, and/or whether to provide legacy content.

In detail, whether or not the work environment setting data is displayed may be information for setting whether to display at least one or more augmented reality contents respectively provided in the corresponding field work.

In an embodiment, the display or not, when an arbitrary field worker enters the field work space (POI) related to the first field work, a plurality of augmented reality content in relation to the first field work corresponds to the field device 100 In the case of augmented display through , it may be information in which whether to delete (stop display) or create (continue display) for each of a plurality of augmented reality contents displayed in augmented reality is set.

In this case, according to an embodiment, the job management application 311 may perform conditional display or not when setting the above-mentioned display in the field device 100 .

In detail, the job management application 311 may provide a conditional display status setting interface to the corresponding field device 100 .

Here, the conditional display setting interface according to the embodiment is based on a predetermined parameter (eg, a parameter based on flow rate, oil pressure, humidity and/or temperature sensing data obtained from a sensor system, etc.), It may be an interface that can set whether to display related augmented reality content provided.

In detail, the field device 100 provided with the conditional display setting interface in the embodiment sets whether to display the augmented reality content based on predetermined parameters (eg, flow rate, hydraulic pressure, humidity and/or temperature parameters, etc.) can do.

In more detail, the field device 100 may determine whether to display the augmented reality content by setting an equation and/or a threshold based on a predetermined parameter.

In an embodiment, the field device 100 may generate work environment setting data in which a predetermined threshold (eg, a preset value, etc.) for the first parameter (eg, a temperature parameter) is set.

And the field device 100 may transmit the work environment setting data set as above to the work management application 311 by including the work data in the work data.

At this time, the job management application 311 that has received the job data from the field device 100 is augmented and displayed on the field device 100 according to a predetermined threshold set for the first parameter based on the received job data. You can decide which augmented reality content to become.

In an embodiment, the task management application 311, when a first parameter value exceeding a predetermined threshold set for the first parameter is detected (eg, a temperature exceeding a predetermined threshold set for the temperature parameter) When sensing data is sensed in real time), the augmented reality content related to the first parameter may be augmented and displayed on the corresponding field device 100 .

In this case, the work management application 311 may set the augmented reality content displayed as above to be displayed on the gaze area of the field worker corresponding to the field device 100 .

In another embodiment, the field device 100 may include a predetermined formula (eg, summing between parameters, etc.) and a threshold (eg, a predetermined value) for a first parameter (eg, a temperature parameter) and a second parameter (eg, a humidity parameter). You can create work environment setting data that set the numerical value of

And the field device 100 may transmit the work environment setting data set as above to the work management application 311 by including the work data in the work data.

At this time, the work management application 311 that has received the work data from the field device 100, based on the received work data, the field device according to predetermined formulas and thresholds set for the first parameter and the second parameter In step 100, it is possible to determine the augmented reality content to be augmented display.

In an embodiment, the work management application 311 may obtain each parameter value for the first parameter and the second parameter in real time.

In addition, the work management application 311 may calculate a sum value of each parameter value obtained in real time.

In addition, the task management application 311, when the calculated sum exceeds the predetermined threshold value, the augmented reality content related to the first parameter and the second parameter to be augmented and displayed on the corresponding field device 100 can

At this time, the work management application 311 provides the augmented reality content to the field device 100 according to the work environment setting data set based on the predetermined formula and/or threshold as above and displays the augmented reality, the formula and / Alternatively, an alarm may be further provided according to the threshold being met.

In addition, the task management application 311 may determine whether to display the augmented reality content based on the conditional display setting interface, based on the sensing data for each parameter based on a predetermined time difference.

In detail, the job management application 311 may provide a conditional display setting interface to the field device 100, and the field device 100 receiving the interface senses the first parameter based on the interface. A predetermined time interval for collecting data can be set.

For example, the field device 100 may collect sensing data for the first parameter every 10 minutes based on a field worker's input to the conditional display setting interface.

In addition, the field device 100 may transmit the work environment setting data set as above to the work management application 311 by including the work data in the work data.

At this time, the job management application 311 that has received the job data from the field device 100 acquires sensing data from the sensor system according to a predetermined time interval set for the first parameter based on the received job data. can do.

At this time, the task management application 311 is augmented to be augmented and displayed on the field device 100 based on a predetermined value obtained by linking a plurality of sensing data for the first parameter collected according to the predetermined time interval. You can set up real content.

In detail, in the embodiment, the work management application 311 may preset a maximum amount of change and/or a minimum amount of change according to a predetermined time interval for the first parameter.

For example, the work management application 311 may preset the maximum amount of change (eg, 50 degrees) according to a predetermined time interval (eg, 10 minutes, etc.) with respect to the temperature parameter, which is the first parameter.

In addition, the work management application 311 may collect sensing data according to a predetermined time interval set in the field device 100 in real time based on the work environment setting data received from the field device 100 in real time.

And the work management application 311, based on a plurality of sensing data values collected according to a predetermined time interval set in the field device 100, and a predetermined maximum change amount of the first parameter, a plurality of sensing data You can perform arithmetic operations involving values.

In addition, the work management application 311, based on a predetermined numerical value derived as a result of the four arithmetic operations performed as above, and a predetermined threshold obtained as work environment setting data from the corresponding field device 100, the field The device 100 may determine augmented reality content to be augmented display.

On the other hand, in the embodiment, the display position of the work environment setting data may be information that sets the display position for each of the at least one or more augmented reality content provided in the corresponding field work.

For example, the display position may be set to a predetermined surrounding area based on the location of the work target object and/or a predetermined area in the field work space (POI).

In addition, in the embodiment, the display method of the work environment setting data may be information setting the shape, size and/or combination of augmented reality content (eg, graph augmented reality content, etc.).

For example, the display method may be information in which a form (eg, a pie graph, a bar graph, and/or a line graph, etc.) for a graph displaying the first sensing data is set.

In another example, the display method may be information in which a combination of sensing data is set to display as one augmented reality content by combining at least some of a plurality of sensing data.

For example, the display method may be information set to display as one graph by combining the first sensing data graph and the second sensing data graph.

In addition, in the embodiment, whether legacy content of the work environment setting data is provided may be information for setting whether to output a parameter value based on an analog instrument panel of the legacy equipment.

That is, in the embodiment, whether legacy content is provided may be information that sets whether to output parameter values based on an analog instrument panel of legacy equipment into augmented reality content.

In detail, in general, legacy equipment may display predetermined parameter values (eg, flow rate, hydraulic pressure, humidity and/or temperature sensing data values, etc.) through an analog instrument panel.

In an embodiment of the present invention, the predetermined parameter value is obtained based on the sensed data sensed by the sensor system, but there may be a case where a difference from the predetermined parameter value output from the analog instrument panel of the legacy equipment occurs. .

Thus, in the embodiment of the present invention, the task management application 311 includes a parameter value of an analog instrument panel of legacy equipment photographed with a camera of a sensor system (eg, a temperature parameter value of an analog instrument panel displaying a temperature parameter, etc.); A parameter value sensed by a sensor (eg, a temperature sensor, etc.) of a sensor system that senses a parameter value corresponding to the parameter value may be compared.

And, when a predetermined difference is detected as a result of the comparison, the task management application 311 may output a predetermined alarm and provide it to the manager.

In addition, in an embodiment, the job management application 311 may receive job data including job environment setting data that sets whether to provide legacy content from the field device 100, and based on the received job data, the job site The augmented reality content provided to the device 100 may be determined.

In detail, the work management application 311 is based on the work data received from the field device 100, the field device 100 determines whether to request the legacy content of the analog instrument panel of the legacy equipment into augmented reality content can do.

For example, the field worker of the field device 100, the data value of the augmented reality content based on the temperature data value of the temperature analog instrument panel of the legacy equipment being worked, and the temperature sensing data provided from the temperature sensor for the legacy equipment. If they are different from each other, it is possible to request legacy content to the task management application 311 by setting whether to provide legacy content.

Thus, the field device 100 may transmit, to the work management application 311 , the job data in which the legacy content provision request for legacy content is set.

In this case, the work management application 311 that has received the work data may determine whether to provide legacy content to the corresponding legacy equipment based on the legacy content provision information of the work data.

In addition, when it is determined to provide legacy content, the task management application 311 may generate legacy content obtained by converting an analog instrument panel of the corresponding legacy equipment into augmented reality content.

In detail, the task management application 311 may generate legacy content by processing the dashboard into 3D augmented reality content based on an image captured by the analog dashboard of the legacy equipment.

In addition, the work management application 311 may transmit the generated legacy content to the corresponding field device 100 to provide augmented display on the gaze area of the field worker.

Alternatively, the task management application 311 may detect a predetermined difference between the parameter value of the analog instrument panel and the parameter value sensed from the sensor system and, by input of the task manager, confirming the provided alarm, for the analog instrument panel of the legacy equipment. Legacy content may be generated, and the generated legacy content may be transmitted and provided to the corresponding field device 100 .

Alternatively, when a predetermined difference between the parameter value of the analog instrument panel and the parameter value sensed from the sensor system is detected, the task management application 311 automatically generates legacy content for the analog instrument panel of the legacy equipment by its own process. Thus, it may be provided to the corresponding field device 100 .

As such, in the embodiment, the task management application 311, when a difference between the parameter value sensed by the analog instrument panel and the parameter value sensed by the sensor system occurs, augmented reality content based on the analog instrument panel and/or the sensor system By selectively generating and providing augmented reality content based on data acquired by sensors of

Meanwhile, in the embodiment, the above work environment setting data may be set in the field device 100 based on an input of a field worker performing field work.

In detail, the field device 100 is a field worker who does not have a separate authority, regardless of the plan given to the field work space (POI) of the corresponding field worker and/or the field device 100 is located. Even in this case, it is possible to set work environment setting data related to arbitrary field work.

Alternatively, the work environment setting data may be automatically generated by a pre-programmed work environment setting data generation process as the corresponding field work progresses.

In addition, in the embodiment, the above work environment setting data may be set in units of a field work space (POI).

In an embodiment, the work environment setting data may be information setting whether to display augmented reality content for each field work space (POI), a display location, a display method, and/or whether to provide legacy content.

Meanwhile, the work performance result data may be information that provides work results derived as field work progresses.

For example, the job performance data includes job completion report data, maintenance performance data, abnormal situation occurrence data and/or feedback generated by input of a field worker who performed the job after completion of field work. Request data and the like may be included.

In an embodiment, the work data including the above work environment setting data and/or work performance result data is generated by a user input of a field worker performing field work, or a pre-programmed work as the field work is in progress It can be created automatically by the data creation process.

In addition, in the embodiment, the management content may be augmented reality-based task management content generated based on the task data obtained from the field device 100 .

In an embodiment, the management content may include process management content, maintenance management content, abnormal equipment management content, and/or feedback management content.

In detail, in an embodiment, the job management application 311 may obtain job data for a job (job) assigned to the field device 100 by interworking with the field device 100 .

10 is an example of input data based on augmented reality in the field device 100 according to an embodiment of the present invention.

In more detail, referring to FIG. 10 , the work management application 311 may receive input data obtained based on augmented reality from the field device 100 .

In an embodiment, the work management application 311 is, from the field device 100, a field image captured by the field device 100, immersive drawing data by the augmented reality-based input of the field worker, and immersive text Data, response input data for an immersive electronic document, and/or immersive work manual creation data may be received as input data.

In addition, in an embodiment, the work management application 311 may receive field worker information and/or corresponding field worker location information at the time of inputting input data from the field device 100 .

And the work management application 311, based on the input data received from the field device 100, field worker information and/or field worker location information, etc., may obtain job data including the data and/or information. .

In addition, in an embodiment, the job management application 311 may create and store management content based on the acquired job data.

That is, the work management application 311, based on the work data received from the field device 100, management content including process management content, maintenance management content, abnormal equipment management content and / or feedback management content, etc. and the created management content can be stored and managed on the workstation service.

In detail, in the embodiment, the work management application 311 may generate 1) process management content based on the obtained work data.

11 and 12 are examples of a state showing process control content according to an embodiment of the present invention.

In detail, referring to FIGS. 11 and 12 , in the embodiment, the work management application 311 may generate process management content, which is content related to a process for managing a process progress state for on-site work and workers.

At this time, in the embodiment, the work management application 311 is a work management graph content that can check the entire work list content and/or real-time process progress status based on the graphic image that can check the entire field work and the process progress by each worker. You can create process control content, including

In this way, the job management application 311, based on the job data received in real time from the field device 100, monitor and manage the process progress for the field work and the operator performing the field work on the workstation service. By creating and utilizing process management contents that can

In addition, in an embodiment, the work management application 311 may generate 2) maintenance management content based on the obtained work data.

In detail, the work management application 311 may generate maintenance management content, which is content related to a maintenance process of a work target object in the field.

In this case, in the embodiment, the work management application 311 may generate maintenance management content including maintenance history (history) content, questionnaire content, and/or visualization content for the work target object.

Here, the maintenance history content for the work target object may provide history information of the maintenance work performed by the field worker with respect to the work target object.

In an embodiment, the work management application 311 may receive the first work data for the first work target object from the field device 100 of the field worker who performed the maintenance work on the first work target object have.

In addition, the work management application 311 may generate maintenance history content for the first work target object based on the received first work data.

For example, the work management application 311 may generate maintenance history content including field worker information, work date and time information, and/or work content record information that performed maintenance work on the first work target object. can

In this case, in the embodiment, the work management application 311 may update and/or generate work history information of the first field worker who performed the maintenance work on the first work target object.

Also, the work management application 311 may calculate the work performance score of the first field worker based on the updated and/or generated work history information of the first field worker.

In addition, the work management application 311 may perform task division and task information provision to the first field worker later on the basis of the calculated task performance score.

For example, the work management application 311, when the work performance score of the first field worker meets a predetermined criterion (eg, more than a preset score, etc.) can be given to workers.

In addition, the survey content for the work target object may provide survey information on the work target object based on the augmented reality content.

In an embodiment, the work management application 311 may transmit an augmented reality-based immersive survey electronic document for the first work target object to the field device 100 .

Also, the work management application 311 may obtain response information according to the field worker input of the field device 100 that has received the transmitted immersive questionnaire electronic document as job data from the field device 100 .

Also, the work management application 311 may generate questionnaire content for the first work target object based on the obtained work data as described above.

In addition, the visualization content for the work target object may be information obtained by visualizing maintenance-related work data obtained from the field device 100 in the form of a graph and/or a dashboard.

In an embodiment, the work management application 311 may obtain work data for the maintenance work performed on the first work target object from the field device 100, and based on the obtained work data, the first work It is possible to create and provide visualization content for the target object.

In this way, the work management application 311, based on the work data received in real time from the field device 100, by generating maintenance management content that can manage the maintenance progress of the work target object in the field, The maintenance history for each target object can be systematically and easily managed based on augmented reality contents.

In addition, the task management application 311 may enable mutual feedback between users based on an augmented reality-based immersive electronic document, and may perform compensation and/or response according to the maintenance result based on this.

In addition, in an embodiment, the work management application 311 may generate 3) abnormal facility management content based on the obtained work data.

In detail, the work management application 311 may generate abnormal equipment management contents that can provide information on abnormal equipment requiring work in the field and manage a maintenance process for the abnormal equipment.

In an embodiment, the work management application 311 may include abnormal equipment notification content, maintenance history content for abnormal equipment, questionnaire content and/or visualization content, etc. in the abnormal equipment management content.

In the following description, descriptions overlapping with those described above among the descriptions of the components of the abnormal facility management content may be summarized and/or omitted.

Here, the abnormal facility notification content according to the embodiment is based on the field worker location information of the job data obtained from the field device 100, the abnormal facility existing in a predetermined surrounding area based on the location of the corresponding field worker. information can be provided along with an alarm.

In an embodiment, the job management application 311 may determine the location of the field worker in real time, based on the job data of the field device 100 obtained in real time.

At this time, the work management application 311, if there is an abnormal facility on a predetermined surrounding area based on the location of the field worker identified in real time, the abnormal facility with the field device 100 of the field worker corresponding to the area information and alerts can be provided.

In addition, in an embodiment, the work management application 311 may generate and provide maintenance history (history) content, questionnaire content, and/or visualization content using an abnormal facility as a work target object.

As such, the work management application 311 provides information on abnormal equipment based on the work data received in real time from the field device 100 and abnormal equipment management content capable of managing maintenance for the abnormal equipment. By providing a field device 100 of a field worker located near an abnormal facility requiring maintenance work, etc., without going through a separate procedure for checking the target object (facility), the abnormal situation for the abnormal facility Occurrence and related information can be provided quickly and clearly based on augmented reality content.

In addition, through this, the work management application 311 can make the repair work of the abnormal equipment occurring in the field proceed quickly and accurately, and the history (history) can also be systematically managed based on the augmented reality content.

In addition, in an embodiment, the work management application 311 may generate 4) feedback management content based on the obtained work data.

13 is an example of a state of showing feedback management content according to an embodiment of the present invention.

In detail, referring to FIG. 13 , in an embodiment, the task management application 311 may generate feedback management content capable of managing a feedback collaboration process between users based on augmented reality content.

In an embodiment, the task management application 311 may include feedback collaboration content in the feedback management content, and maintenance history content, survey content and/or visualization content for field work performed based on the feedback collaboration content, etc. may further include.

In the following description, among the descriptions of the components of the feedback management content, descriptions that overlap with those described above may be summarized and/or omitted.

Here, the feedback collaboration content according to the embodiment includes a process for requesting a predetermined advice on a task given by a field worker, and a user (in the embodiment, a remote administrator) who can provide feedback as a response to the request. You can implement a process that provides guidance for that request.

In detail, in the embodiment, the work management application 311 may obtain, from the field device 100 , feedback request information in which the field worker requests advice on a specific job as job data.

In this case, exemplary feedback request information includes a field image recorded from the field device 100 of the field worker and/or information that the field worker requests for advice on a specific task (eg, inquiry text, etc.) It may be work data.

In addition, in an embodiment, the work management application 311 may generate and/or obtain feedback providing information that provides a response (feedback) to the acquired feedback request information.

For example, the work management application 311 may generate and/or obtain feedback providing information including a guidance text for feedback request information, an attached file, and/or a related manual.

In addition, in an embodiment, the work management application 311 may generate feedback collaboration content based on the feedback request information and the feedback provision information obtained as described above.

In this case, the work management application 311 may share the generated feedback collaboration content with the field device 100 to implement remote collaboration communication.

In addition, in the embodiment, the work management application 311 generates and provides maintenance history (history) content, survey content and/or visualization content related to field work and work target object performed based on the feedback collaboration content as above. can do.

In this way, the work management application 311, based on the work data received in real time from the field device 100, generates and provides feedback management content that can manage the feedback collaboration process between users based on the augmented reality content By doing so, the remote manager can more easily and intuitively understand the inquiries of field workers through realistic augmented reality contents, and the feedback is also produced and provided as augmented reality contents. You can more conveniently understand the guidance information and carry out work.

That is, in the embodiment, the work management application 311 is based on real-time work data obtained from the field device 100, augmented reality-based management content (process management content, maintenance management content, abnormal facility management content and / Alternatively, feedback management content, etc.) may be generated, and the generated immersive management content may be shared among users on a workstation service.

At this time, the task management application 311 may enable remote communication based on augmented reality content between a plurality of users on a workstation service by sharing the generated management content with at least one field device 100, and through this It can improve the accuracy and convenience of field work.

Here, the management content on the augmented reality screen output from the field device 100 may be augmented and provided at a predetermined location around the work target object matching the management content. A detailed description thereof will be provided later.

14 is an example of generating sensing content based on sensing data according to an embodiment of the present invention.

Also, referring to FIG. 14 , in an embodiment, the work management application 311 may obtain sensing data from the on-site sensor system 400 and may generate sensing content based on the obtained sensing data. (S107)

In detail, the work management application 311 may obtain sensing data for the work target object from the field sensor system 400 .

Here, the sensing data according to the embodiment may include image sensing data, flow rate sensing data, hydraulic pressure sensing data, humidity sensing data, and/or temperature sensing data obtained through sensing of a work target object.

In this case, in the embodiment, the work management application 311 may acquire real-time sensing data before, during, and/or upon completion of the work on the work target object.

In addition, the task management application 311 may generate sensing content that is augmented reality content based on the acquired sensing data.

In detail, in the embodiment, sensing content may be augmented reality content in which sensing data obtained from the on-site sensor system 400 is converted into an augmented reality-based graph or dashboard.

For example, the sensing content may include graphed sensing data (ie, a bar graph, a line graph, and/or a pie graph generated based on a sensing data-based operation) and/or dashboarded sensing data. can

At this time, in the embodiment, when generating sensing content, the task management application 311 is based on the properties of the acquired sensing data, a graph type, scale, and/or threshold (in the embodiment) for the sensing data. , an alarm threshold for providing an alarm when a predetermined threshold is exceeded, etc.) can be set.

As such, the work management application 311 provides augmented reality content by reflecting the sensing data obtained in conjunction with the field sensor system 400, thereby providing more diverse and detailed information necessary for the work of field workers. and, through this, it is possible to improve the safety and reliability of performing field work.

In addition, in an embodiment, the task management application 311 may set mapping information for augmented reality content including the generated management content and sensing content. (S109)

Here, the mapping information according to the embodiment may be information for determining a virtual location where the augmented reality content is to be augmented on an augmented reality screen (ie, a field image and/or a three-dimensional site map viewed by a field worker).

In detail, in the embodiment, the task management application 311 is a work target object that matches each of the augmented reality content including the management content and the sensing content by interworking with the workstation server system 500 and/or the field device 100, etc. A target object (target facility) can be detected.

In addition, the work management application 311 may acquire the actual location information of the detected target object at the work site through interworking with the workstation server system 500 and/or the field device 100 .

In addition, in an embodiment, the task management application 311 may determine a virtual location, which is a location coordinate to augment and display augmented reality content in response to a corresponding target object in a virtual environment, based on the obtained actual location information of the target object. have.

In an embodiment, the task management application 311 is an augmented display position of augmented reality content that matches a predetermined area (eg, within a radius of 30 cm, etc.) based on the actual position of the first target object to the first target object. It can be determined as a virtual location.

In addition, in the embodiment, the work management application 311 is an augmented reality screen (ie, a field image and/or a three-dimensional site map, etc. viewed by a field worker) for an actual location of the job site that matches one virtual location. When the field device 100 is detected in a predetermined area, augmented reality content matching the area may be provided to the detected field device 100 .

In addition, in an embodiment, the task management application 311 may provide a real-time remote collaboration service based on augmented reality content including management content and sensing content. (S111)

In detail, the task management application 311 may provide a remote collaboration service for implementing remote collaboration between users based on augmented reality based on the augmented reality content and mapping information for each corresponding augmented reality content.

15 is a flowchart illustrating a method of performing real-time remote collaborative communication based on augmented reality content according to an embodiment of the present invention.

In more detail, referring to FIG. 15 , in the embodiment, the work management application 311 provides the augmented reality content for the work target object to the field device 100 of the field worker who is assigned the work for the corresponding work target object. can provide (S201)

In an embodiment, the work management application 311 may detect, from the database, maintenance management content that is augmented reality content for implementing a maintenance process for the work target object.

In this case, in the embodiment, the work management application 311 may acquire sensing data for the first field work space (POI), and based on the acquired sensing data, field work related to the first field work space (POI) It can be determined whether the creation of

And when it is determined that the work management application 311 requires on-site work for the first field work space (POI), augmented reality content that implements a maintenance process for the work target object in the first field work space (POI) (maintenance management content) can be detected from the database.

In addition, the work management application 311, the maintenance management content, which is the detected augmented reality content, is given a maintenance work for the work target object, or the site of the field worker who has entered the first field work space (POI). It may be provided by transmitting to the device 100 .

In this case, according to an embodiment, the task management application 311 may further include guide information for explaining the guide information for the transmitted augmented reality content.

For example, when the work management application 311 transmits the maintenance management content to the field device 100, basic information (eg, the work target object name, function) about the work target object targeted by the maintenance management content , shape, size, weight and/or location information, etc.) may be further provided as guide information.

In an embodiment, the augmented reality content and/or guide information provided as above is on the augmented reality screen (scene image and/or 3D site map, etc.) of the field device 100 based on the mapping information of the corresponding augmented reality content. can be displayed in

In addition, in an embodiment, the work management application 311 may receive work data from the field device 100 that has received the augmented reality content. (S203)

That is, the job management application 311 may obtain job data for a job performed based on the transmitted augmented reality content from the field device 100 that has received the augmented reality content.

In an embodiment, the work management application 311 may transmit the maintenance management content for the work target object to the first field device 100, and the operation (process, task) performed based on the maintenance management content ) may receive work data for the first field device 100 .

In addition, in an embodiment, the job management application 311 may generate management history information based on the received job data. (S205)

Here, the management history information according to the embodiment is base data for generating and/or updating augmented reality content, and may be one-dimensional management information built on a workstation service based on work data.

In an embodiment, the management history information may include operation guidance information, maintenance management information, abnormal facility management information, sensing information, and/or feedback management information based on operation data.

In detail, in an embodiment, the work management application 311 may acquire work data for a work (process, task) performed based on the maintenance management content for the first work target object.

In addition, the work management application 311 may generate maintenance management information for the first work target object based on the obtained work data.

Also, the work management application 311 may update and/or generate management history information including the generated maintenance management information.

In addition, in an embodiment, the task management application 311 may convert the generated management history information into augmented reality content. (S207)

In detail, the task management application 311 may convert the generated management history information into augmented reality content and convert it into a form capable of displaying a three-dimensional augmentation.

In an embodiment, the work management application 311 may generate maintenance management content by converting the maintenance management information of the generated management history information into augmented reality content.

In addition, in an embodiment, the task management application 311 may perform an update on the existing augmented reality content based on the augmented reality content generated by converting the management history information. (S209)

That is, the work management application 311 may perform an update of the existing augmented reality content by reflecting the newly generated augmented reality content based on the work data newly acquired from the field device 100 .

In an embodiment, the work management application 311 may update the existing maintenance management content by reflecting the newly created maintenance management content.

As such, the task management application 311 provides a remote collaboration process based on the augmented reality content that is updated and/or generated in real time, thereby providing a customized process management service, augmented reality content link service, maintenance management service, and augmented reality based It is possible to improve the quality of workstation service that can remotely and collectively perform services necessary for overall management of field work and workers, such as remote collaboration service and/or feed-based process management service of It is possible to easily and easily perform content-based on-site management and guidance, and at the same time, more accurately and systematically manage the process progress across industrial sites.

Above, augmented reality content-based task management system and method according to an embodiment of the present invention, by implementing a remote collaboration service based on the augmented reality content, remotely based on the augmented reality content on-site management and provision of guidance (Guidance) There is an effect that can be done.

In addition, the augmented reality content-based work management system and method according to an embodiment of the present invention, by implementing a workstation service capable of managing a user (ie, a worker) and the corresponding work remotely located at the work site, the overall industrial site It has the effect of being able to easily control the progress from a macro perspective.

In addition, the augmented reality content-based task management system and method according to an embodiment of the present invention implements collaborative communication between users (ie, managers and workers) located remotely from each other based on a virtual environment, thereby creating augmented reality content Through the realistic screen, it is possible to provide easy and clear work guidance to remote field workers, which has the effect of enabling workers to understand and perform work more intuitively.

In addition, the augmented reality content-based task management system and method according to an embodiment of the present invention generates or updates augmented reality content based on real-time data generated through the execution of a task, thereby providing remote collaborative communication in real time At the same time, it has the effect of improving the safety of work by improving the accuracy of the information provided to workers at the industrial site while improving the quality of the work.

In addition, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention or may be known and used by those skilled in the computer software field. Examples of computer-readable recording media include hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floppy disks. medium), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. A hardware device may be converted into one or more software modules to perform processing in accordance with the present invention, and vice versa.

The specific implementations described in the present invention are only examples, and do not limit the scope of the present invention in any way. For brevity of the specification, descriptions of conventional electronic components, control systems, software, and other functional aspects of the systems may be omitted. In addition, the connection or connection members of lines between the components shown in the drawings illustratively represent functional connections and/or physical or circuit connections, and in an actual device, various functional connections, physical connections that are replaceable or additional may be referred to as connections, or circuit connections. In addition, unless there is a specific reference such as “essential” or “importantly”, it may not be a necessary component for the application of the present invention.

In addition, although the detailed description of the present invention has been described with reference to a preferred embodiment of the present invention, those skilled in the art or those having ordinary knowledge in the art will appreciate the spirit of the present invention described in the claims to be described later. And it will be understood that various modifications and variations of the present invention can be made without departing from the technical scope. Accordingly, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be defined by the claims.

Claims (22)

As a method for remotely managing field work based on augmented reality content by a work management application executed by a workstation device,
building a workstation and creating a workspace;
providing tasks for each field device of field workers included in the workspace based on the augmented reality contents;
obtaining work data according to the performance of the field work from the field device;
generating augmented reality-based management content based on the obtained work data;
acquiring sensing data for the work target object of the field work from the field sensor system of the field;
generating augmented reality-based sensing content based on the acquired sensing data;
setting mapping information for determining a virtual location where the augmented reality content including the management content and the sensing content is to be augmented on the augmented reality screen viewed by the field worker; and
Comprising the step of performing remote collaborative communication based on the augmented reality content and the mapping information
Augmented reality content-based task management method. The method of claim 1,
The workstation is
A web or mobile service environment that implements a workstation service that remotely manages the field work and the field worker
Augmented reality content-based task management method. 3. The method of claim 2,
The workstation service is
It provides at least one service among customized process management service, augmented reality content link service, maintenance management service, augmented reality-based remote collaboration service, and feed-based process management service.
Augmented reality content-based task management method. 3. The method of claim 2,
The workspace is
On the workstation, the user of the workstation service invites and organizes business personnel related to the field work,
Augmented reality content-based task management method. The method of claim 1,
Creating the workspace includes:
Inviting at least one or more field workers as members to the workspace, and allocating a plan for each field worker.
Augmented reality content-based task management method. 6. The method of claim 5,
The plan is
Information that sets the service use right and service use period for the workspace
Augmented reality content-based task management method. 7. The method of claim 6,
The right to use the service is
Containing at least one or more of a Make permission to produce the augmented reality content, a View permission to implement visualization, and a Remote permission to perform remote collaborative communication
Augmented reality content-based task management method. The method of claim 1,
The augmented reality content is,
It is a content that is augmented in a three-dimensional virtual environment and provides information related to the field work.
Augmented reality content-based task management method. The method of claim 1,
The step of assigning a task for each field device is,
Including the step of providing business information for each field device,
The business information is
Information that explains or assists the task given to the field worker, including at least one of worker designation information for the task, work target object designation information, task content information, and related augmented reality content information
Augmented reality content-based task management method. 8. The method of claim 7,
The step of assigning a task for each field device is,
Based on the plan, comprising the step of limiting the right to use the augmented reality content related to the work for each field device
Augmented reality content-based task management method. The method of claim 1,
The step of obtaining the work data includes:
acquiring augmented reality-based input data from the field device;
Comprising the step of acquiring at least one of operator information about the field worker who has input the input data and the location information of the field worker when the input data is input
Augmented reality content-based task management method. The method of claim 1,
The management content is
Containing at least one of process management content, maintenance management content, abnormal equipment management content, and feedback management content
Augmented reality content-based task management method. The method of claim 1,
The step of acquiring the sensing data includes:
Comprising the step of acquiring real-time sensing data before, in progress, or completion of the on-site work for the work target object
Augmented reality content-based task management method. 14. The method of claim 13,
The sensing content is
Augmented reality content generated by converting the sensing data into the form of an augmented reality-based graph or dashboard
Augmented reality content-based task management method. The method of claim 1,
The step of performing the remote collaborative communication comprises:
providing the augmented reality content for the work target object to the field device of a field worker assigned the task for the work target object;
Receiving the work data from the field device provided with the augmented reality content;
generating management history information based on the received job data;
converting the generated management history information into augmented reality content;
Comprising the step of updating the augmented reality content based on the management history information of the augmented reality content
Augmented reality content-based task management method. 16. The method of claim 15,
The step of providing the augmented reality content to the field device of the field worker who has been given the task for the work target object,
Comprising the step of further providing guide information to explain the guide information for the augmented reality content
Augmented reality content-based task management method. 17. The method of claim 16,
Displaying the augmented reality content or the guide information on the augmented reality screen output from the field device based on the mapping information for the augmented reality content
Augmented reality content-based task management method. 16. The method of claim 15,
The management history information is
It is a base data that is generated based on the work data and generates or updates the augmented reality content.
Augmented reality content-based task management method. The method of claim 1,
The step of assigning a task for each field device is,
dividing the space in which the field work is performed into at least one or more point of interest (POI);
Comprising the step of assigning the task to each of the divided field work spaces
Augmented reality content-based task management method. 20. The method of claim 19,
setting a part of the at least one field work space as a safety management area;
When the field worker enters the set safety management area, further comprising the step of providing a predetermined alarm to the field device of the field worker
Augmented reality content-based task management method. The method of claim 1,
The step of obtaining the work data includes:
acquiring work environment setting data, which is information for setting at least one of whether to display the augmented reality content, a display position, a display method, and whether to provide legacy content;
Comprising the step of setting the augmented reality content to be augmented display in the field device based on the obtained work environment setting data
Augmented reality content-based task management method. 22. The method of claim 21,
The work environment setting data is
The on-site device or the space in which the on-site work is performed is divided into at least one or more areas and is set for each of at least one or more on-site workspaces
Augmented reality content-based task management method.

Images