KR102579050B1 - System and method for knowledge service - Google Patents

Abstract

The present invention relates to a knowledge service provision system and method. The present invention provides work information in the form of augmented reality content through smart glasses to workers in various industrial sites, so that work capabilities can be increased by increasing understanding of facilities, devices, and processes, and the production and supply of augmented reality content is provided on the web. By providing it as a base system, the author's convenience is taken into consideration, and data usability is increased by linking with the customer's legacy system. The system is easy to update and manage, so it can be used in a variety of fields.

Description

Knowledge service provision system and method {SYSTEM AND METHOD FOR KNOWLEDGE SERVICE}

The present invention relates to a system and method for providing knowledge services, and more specifically, to a system and method for providing knowledge services using augmented reality content.

In industrial facilities, inspectors perform safety inspections with equipment worn for safety inspections, equipment carried for measurement, and thick and complicated inspection logs, which causes inconvenience to inspectors performing safety inspections.

Meanwhile, augmented reality (augmented reality) refers to a computer graphics technique that synthesizes virtual objects or information in the real environment to make them look like objects that exist in the original environment. When applied to the safety inspection of industrial facilities described above, augmented reality technology can provide much convenience to inspectors. There is a demand for the development of a knowledge service system using augmented reality technology in various fields, including the above-mentioned industrial facilities.

Meanwhile, chemical plants are composed of very diverse and complex facilities and devices, and handle a lot of harmful chemicals, so even a small accident can have a big impact on workers and the surrounding environment, so the Occupational Safety and Health Act, Chemical Substances Control Act, and High Pressure Gas Safety It is subject to strict application of various regulations such as the Management Act and the Dangerous Materials Safety Management Act. To prevent chemical accidents, chemical plants carry out various facility preventive inspection activities and manage facilities and devices. In order to minimize accident damage in the event of an accident, various facilities and devices such as hazardous substance detection devices and prevention facilities are built. , emergency response medicines are available. However, above all, in order to prevent the spread of chemical accident damage and minimize its impact, the capabilities of field workers responsible for initial response are very important. Accordingly, chemical plants have formed an emergency response organization, created an emergency response manual to respond to accidents, and are conducting regular education and training. Field workers working at chemical plants must acquire a vast amount of information, including professional skills for operation (normal and emergency operation) and inspection of factory facilities, information on the materials being handled, and response information in the event of an accident. For this purpose, you must wear various personal protective equipment (protective clothing, face shield, mask, gloves, safety shoes, etc.), and depending on the work, you must carry thick documents and heavy equipment while working. In this work environment, when an emergency situation such as a chemical accident occurs, it is difficult for novice field workers without much experience to respond quickly.

The background technology for the present invention has been published in Republic of Korea Patent Publication No. 10-2009-0056760.

The present invention provides a knowledge service provision system and method that provides knowledge information necessary for efficient work performance of workers in various fields in the form of augmented reality content through smart glasses.

In addition, the present invention provides a knowledge service provision system and method that allows a knowledge service provider to easily produce and supply augmented reality content necessary for efficient work performance of workers.

The present invention provides routine work information required for on-site work by field workers in industrial sites, such as chemical plants, that are comprised of and operated with very diverse and complex facilities and devices, as well as information that can be used to prevent malfunctions of facilities or devices or accidents (leakage or leakage of chemical substances). , fire, and explosion) provides a knowledge service system and method that provides emergency response information in real time to field workers through wearable devices according to worker skill level and location.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned can be clearly understood from the description below.

According to one aspect of the present invention, a knowledge service provision system is provided.

The knowledge service provision system according to an embodiment of the present invention provides an augmented reality content authoring tool that creates augmented reality content by linking objects, markers, and information, and the authored augmented reality content through communication with a wearable device and provides the wearable device. It may include a knowledge service providing server that supports the activities of.

According to another aspect of the present invention, a method for providing knowledge services and a computer program for executing the same are provided.

A method of providing knowledge services according to an embodiment of the present invention includes the steps of detecting the occurrence of an event, the step of confirming the occurrence area of the event detected through a sensor or camera connected through wired or wireless communication, and the event type is emergency maintenance of a facility or equipment. If necessary, check the location of the emergency maintenance worker, select the worker to be assigned to emergency maintenance based on the location confirmation result, instruct the selected worker to perform emergency maintenance, and instruct the next-highest priority worker to provide emergency maintenance support. It may include steps of performing and receiving a report of completion of emergency repairs.

According to an embodiment of the present invention, work capabilities in the form of augmented reality content can be provided to workers at various industrial sites through smart glasses to increase understanding of facilities, devices, and processes, thereby increasing work capabilities.

According to an embodiment of the present invention, the production and supply of augmented reality content is provided through a web-based system, so that the convenience of the author is taken into consideration, data usability is increased by linking with the customer's legacy system, and system updates and management are possible. It is easy to use and can be used in various fields.

According to an embodiment of the present invention, in an industrial site, such as a chemical plant, which is comprised of and operated with very diverse and complex facilities and devices, field workers receive routine work information required for field work and failure of facilities or devices or accidents (chemical Emergency response information for emergency situations such as material leakage or leakage, fire, or explosion accident) can be provided to field workers in real time through wearable devices depending on the worker's skill level and location.

1 to 2 are diagrams for explaining a knowledge service providing system according to an embodiment of the present invention.
3 to 11 are diagrams for explaining a knowledge service providing server according to an embodiment of the present invention.
12 to 14 are diagrams for explaining a method of providing a knowledge service according to an embodiment of the present invention.
15 to 21 are diagrams for explaining a method of providing a knowledge service according to another embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail through detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention. In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings. In order to facilitate overall understanding in describing the present invention, the same reference numbers will be used for the same means regardless of the drawing numbers.

1 and 2 are diagrams for explaining a knowledge service provision system according to an embodiment of the present invention.

Referring to FIGS. 1 and 2 , the knowledge service providing system includes an augmented reality authoring terminal 100, a knowledge service providing server 200, and smart glasses 300.

The augmented reality authoring terminal 100 creates augmented reality content using objects, markers, and knowledge information registered in the knowledge service providing server 200. Here, an object is an object (person, object, etc.) recognized through a wearable device such as smart glasses. A marker is a medium for recognizing an object and may be, for example, at least one of a barcode, QR code, text, image, and location coordinates. Knowledge information is valuable information provided through wearable devices and is mapped to objects and markers.

The knowledge service providing server 200 registers objects, markers, and information for authoring augmented reality content, and provides a platform on which augmented reality content required for knowledge service can be easily authored through the augmented reality authoring terminal 100. The knowledge service providing server 200 may be linked with a customer legacy system to create augmented reality content for the augmented reality authoring terminal 100. The knowledge service providing server 200 may support the activities of the wearable device wearer by providing augmented reality content created through communication with the wearable device. Here, the activity is a series of activities performed through a wearable device, such as information inquiry, work, education, and experience, and is created through a combination of augmented reality content. The configuration and function of the knowledge service providing server 200 will be described in more detail in FIGS. 3 to 11 below.

The wearable device 300 receives augmented reality content through communication from the knowledge service providing server 200 and performs the wearer's activity. The wearable device 300 may be linked with a separate communication terminal, for example, a smartphone, for communication with the knowledge service providing server 200, and may itself include a communication module. The wearable device 300 may be, for example, smart glasses, but is not limited thereto and may include any electronic device that can be worn on a person's body and perform an activity. Hereinafter, the wearable device 300 will be described using smart glasses as an example.

Smart glasses 300 can perform specific activities such as safety inspection management at industrial sites, counter-terrorism response at social sites, or vehicle maintenance training at educational sites. The smart glasses 300 may include menus and functions as shown in Table 1 below to perform the specific activities described above.

Menus and functions of smart glasses division explanation log in Smart glasses user application access
- Input through voice recognition
- Text input main function Activities List Provides a list of activities stored in or linked to smart glasses Recognition module Selection of recognition module for marker recognition through smart glasses
- Object recognition
- Face recognition
- Car license plate recognition
- Bar code/QR code recognition
- Character recognition
- Voice recognition Activity Results When a user performs an activity through smart glasses, information such as saved data or files (photos, videos, voice recordings) is provided. video recording Video recording using the camera function within smart glasses Photo shoot Take photos using the camera function in smart glasses voice recording Voice recording function of smart glasses setting Whether to use communication function
Smart glasses display transparency settings
Smart glasses video quality settings
Smart glasses photo quality settings

3 to 11 are diagrams for explaining a knowledge service providing server according to an embodiment of the present invention.

Referring to FIG. 3, the knowledge service providing server 200 includes a main home 210, a business module 220, and an operator module 230.

The main home 210 includes a search menu 212, a site map 214, and a guide 216.

The business module 220 includes a knowledge information storage unit 222, a wearable activity unit 224, an augmented reality studio 226, a statistics unit 228, a user management unit 230, and a my page unit 232.

The knowledge information storage unit 222 registers object information that is a recognition target and marker information for object recognition. Here, marker information can register a plurality of markers in one object, and various types of markers can be registered in one marker. Here, the marker type may be any one of text, bar code, QR code, and image. The knowledge information storage unit 222 recognizes objects through the smart glasses 300 and registers the information provided. The knowledge information storage unit 222 may provide input functions (editing, tables, inspection items, etc.) according to the information type to store information.

The wearable activity unit 224 registers activities and provides modification, deletion, or inquiry functions. The wearable activity unit 224 provides a list of registered activities, and you can set them as your activities through bookmarking activities in the list. In addition, the wearable activity unit 224 can register information (e.g., work information and files, etc.) registered in the smart glasses 300 by a user wearing the smart glasses 300 to the system, and can register the activities registered by the smart glasses user. Result information can be provided. In addition, the wearable activity unit 224 searches activity results and provides a download function as data in a spreadsheet format.

The augmented reality studio 226 is an augmented reality content authoring tool that links objects, markers, and information. Augmented Reality Studio 226 provides event management and 3D view (GUI) setting functions. Additionally, augmented reality studio 226 may provide a list of authored augmented reality content.

The statistics unit 228 provides statistical information of activities and statistical information of knowledge information.

The user management unit 230 registers users who wish to use the knowledge service and provides a search function. The user management unit 230 may further provide user system access history information.

The My Page unit 232 provides a list of activities available to the user. The My Page unit 232 can provide and modify registered user information. Specifically, the My Page unit 232 can register (upload) information (e.g., work information, files, etc.) registered in the smart glasses 300 by a user wearing the smart glasses 300 within the system. Additionally, the My Page unit 232 provides activity result information registered by the smart glasses user.

The operator module 240 includes a category management unit 242, a wearable device unit 244, an authority management unit 246, a menu management unit 248, a code management unit 250, and an application version management unit 252.

The category management unit 242 provides category information such as objects, activities, and organizations, and provides registration, modification, and deletion functions.

The wearable device unit 244 provides registered smart glasses information and provides device registration, modification, and deletion functions.

In addition, the wearable device unit 244 provides an activity management (registration/modification/delete) function available for each smart glasses device and provides a list of activities downloaded in the smart glasses device.

The authority management unit 246 sets the authority to use the system menu.

The menu management unit 248 provides a system menu list, for example, log information for each menu.

The code management unit 250 provides a list of system codes.

The application version management unit 252 manages smart glasses application version information. The application version management unit 252 registers and distributes updated applications and provides distribution history.

Referring to FIG. 4, the knowledge service providing server 200 provides a main home screen that takes user convenience into consideration. Here, the main home screen is the wearable activity menu (410), knowledge information storage menu (420), augmented reality studio menu (430), statistics menu (440), system usage guide menu (450), and manual download menu (460). ) and system menu information provision (470).

Referring to FIG. 5, the object registration menu of the knowledge information storage menu 420 includes a node tree map 510 that provides object-marker-information registered by the user in a tree structure, and an object tab (Tab), which is an object information registration screen. 520), register information about the object by entering object code, object name, object category, and object description (530), add image file to attach image file (540), upload progress rate and file capacity providing image file upload progress, and It provides functions to store file capacity (550) and input object information (560).

Referring to FIG. 6, the marker registration menu of the knowledge information storage menu 420 includes a node tree map 610 that provides object-marker-information registered by the user in a tree structure, and a marker tab (Tab), which is a marker information registration screen. 620), marker name, marker description, marker location, marker address (when selecting an address, register using the zip code search function, enter latitude, longitude, altitude location information, select from text, barcode, QR code, and image) It provides functions to input information (630), add an image file to attach an image file (640), upload progress and file capacity (650), and save the entered marker information (660). .

Referring to FIG. 7, the information registration menu of the knowledge information storage menu 420 includes a node tree map 710 that provides object-marker-information registered by the user in a tree structure, and an information (Tab) 720 that is an information registration screen. ), selection of information registration tool according to the type of registration information (730), input of related marker and title (740), information input screen (750), storage of entered information (760), and all information including objects, markers, and information. After entering, a registration completion (770) function is provided.

Referring to FIG. 8, the activity list menu of the wearable activity menu 410 includes an activity category, an activity ID, an activity name, an activity search condition 810 for selecting whether to activate the activity, and an activity list 820 that provides a list of registered activities. ), individual activity selection, selection of all activities (830), and a bookmark function (840) for selecting the activity the user wants to use among registered activities.

Referring to FIG. 9, the activity list menu of the wearable activity menu 410 includes an activity (Tab) 910, an activity code, an activity name, and whether the activity is activated. Activity information input screen 920 including activity category, expiration date, version, cycle (optional), cycle unit, and start time (optional), function to save the entered activity information (930) and retrieve existing information (940) provides.

Referring to FIG. 10, the activity list menu of the wearable activity menu 410 includes augmented reality content (Tab) 1010, a list of all registered augmented reality content 1020, and a list of augmented reality content selected to be provided for the activity 1030. ), setting the order (activity scenario) of the selected augmented reality content (1040), providing an activity scenario according to the order of the selected augmented reality content (1050), and storing the set augmented reality content information (1060).

Referring to FIG. 11, the activity list menu of the wearable activity menu 410 has a search function for smart glasses 300 that will provide the corresponding activity among the wearable device (Tab) 1110 and smart glasses 300 registered in the system. After completing registration (1120), activity (basic information), augmented reality content, and wearable device information, click Registration Complete to register activity information in the system (1130).

Figures 12 to 14 are diagrams for explaining a method of providing knowledge services according to an embodiment of the present invention.

Referring to FIG. 12, in steps S1205 and S1210, the knowledge service providing server 200 receives a version check request from the smart glasses 300 and provides version information. The smart glasses 300 can be linked to a communication terminal such as a smartphone or communicate with the knowledge service providing server 200 using Wi-Fi or a mobile communication network through its own communication module. Smart glasses 300 can run and load an application and check the latest version when starting the application.

In steps S1215 and S1220, the knowledge service providing server 200 receives a request for an activity list from the smart glasses 300 and provides it.

In steps S1225 to S1235, the knowledge service providing server 200 receives a request to download activity content from the smart glasses 300, provides the download, and receives download status information from the smart glasses 300.

In steps S1240 and S1245, the knowledge service providing server 200 receives a login request from the smart glasses 300 and responds. The smart glasses 300 can input the user ID using voice or a keypad text input tool.

In steps S1250 and S1255, the knowledge service providing server 200 receives a marker inquiry request from the smart glasses 300 and responds. The smart glasses 300 recognize the marker of the recognition target through the recognition module and request marker inquiry. Here, the marker can be any one of a barcode, QR code, text, and image of an object or person, and if it is not recognized, the information can be retrieved through the voice recognition function. The smart glasses 300 can check corresponding augmented reality content information by searching the marker of the recognition target.

In steps S1260 and S1265, the knowledge service providing server 200 receives a request to upload activity results from the smart glasses 300 and responds. The smart glasses 300 can upload activity result data, such as video shooting, photo shooting, or voice recording files, to the knowledge service providing server 200 by moving to a communication area.

Referring to FIG. 13, the knowledge service providing server 200 can be applied to various industrial fields such as manufacturing, logistics, and medical care. The knowledge service provision server (200) provides information such as factory equipment, chemicals, hazards, driving manuals, safety rules, emergency response, and laws necessary for field work to field workers in real time through smart glasses (300). It can support facility inspection and monitoring of chemical leaks. In addition, the knowledge service provision server 200 can support two-way communication with managers or experts for emergency response and accurate decision-making, and can enable accurate and safe field work even for non-skilled workers.

Referring to FIG. 14, the knowledge service providing server 200 can provide knowledge services for police work by utilizing the recognition technology of the smart glasses 300 for the purpose of public safety in the public sector. The knowledge service providing server 200 can search for information on wanted or stolen vehicles, search for information on suspicious behavior suspects, and search for information through license plate recognition in police vehicles. In addition, the knowledge service provision server 200 may review trial application only in special fields such as provision of terrorist information for counter-terrorism.

15 to 21 are diagrams for explaining a method of providing a knowledge service according to another embodiment of the present invention.

Referring to FIG. 15, in step S1510, the wearable device 300 receives a work instruction message from the knowledge service providing server 200.

In step S1520, the wearable device 300 receives equipment for work preparation from the knowledge service providing server 200 and outputs them. Here, the equipment for work preparation may be at least one of personal protective equipment and a communication terminal.

In step S1530, the wearable device 300 receives work location information from the knowledge service providing server 200 and outputs route guidance information.

In step S1540, the wearable device 300 calls corresponding content when a work facility or equipment is recognized.

Referring to FIG. 16, the wearable device 300 can recognize work facilities or equipment through voice recognition, 2D or 3D image recognition, or character recognition such as text, barcode, or QR code.

In step S1550, the wearable device 300 outputs work content for the work facility or equipment, and the corresponding work is performed based on the work content.

Referring to FIG. 17 , the wearable device 100 may perform a task of guiding work facilities or equipment by work content.

Referring to FIG. 18, in step S1810, the wearable device 300 receives a work instruction message from the knowledge service providing server 200.

In step S1820, the wearable device 300 receives equipment for work preparation from the knowledge service providing server 200 and outputs them. Here, the equipment for work preparation may be at least one of personal protective equipment and a communication terminal.

In step S1830, the wearable device 300 receives work location information from the knowledge service providing server 200 and outputs route guidance information.

In step S1840, when the work facility or equipment is recognized, the wearable device 300 calls the corresponding content and performs an inspection task.

In step S1850, the wearable device 300 stores inspection work result information. Here, the inspection work result information can be stored through inspection value input, photo taking, or voice memo.

In step S1860, the wearable device 300 transmits the stored work result information to the knowledge service providing server 200. The wearable device 300 automatically transmits the inspection result information to the knowledge service providing server 200 when communication is possible, and when the wearable device 300 changes to a communication-enabled state when communication is not possible, for example, communication in an area where communication is not possible. When moving to an available area, inspection result information can be transmitted to the knowledge service provision server 200.

Referring to FIG. 19, the knowledge service providing server 200 may receive work result information received from the wearable device 300 and store it in a database.

Referring to FIG. 20, in step S2005, the knowledge service providing server 200 detects the occurrence of an event. The knowledge service providing server 200 can detect the occurrence of an event by analyzing images input from a sensor or camera connected through wired or wireless communication.

In step S2010, the knowledge service providing server 200 confirms the occurrence area of the event detected through a sensor or camera connected through wired or wireless communication.

In step S2015, the knowledge service providing server 200 checks the event type.

In step S2020, the knowledge service providing server 200 checks the location of the emergency maintenance worker if the event type requires emergency maintenance of the facility or equipment. Here, the worker's location can be confirmed by referring to the attendance record information and work schedule information database.

In step S2025, the knowledge service providing server 200 selects a worker to be deployed for emergency maintenance based on the location confirmation result. As a result of checking the location of the worker, the knowledge service providing server 200 may select the worker located closest to the emergency maintenance facility or installation, or select the worker who is not scheduled to work.

In step S2030, the knowledge service providing server 200 may instruct the selected worker to perform emergency maintenance and the next-ranked worker to provide emergency maintenance support.

In step S2035, the knowledge service providing server 200 receives an emergency maintenance completion report from the selected worker after emergency maintenance is performed. The knowledge service provision server 200 recognizes facilities or equipment subject to emergency maintenance by the wearable device of the selected worker, outputs emergency maintenance content corresponding to the facility or equipment subject to emergency maintenance, and performs emergency maintenance based on the content. After completion of emergency repairs, you can receive a report.

Meanwhile, in step S2040, if the type of event is an accident, the knowledge service providing server 200 confirms the worker's location in order to respond to the accident. Here, the worker's location can be confirmed by referring to the attendance record information and work schedule information database.

In step S2045, the knowledge service providing server 200 instructs the emergency response command to the worker of the emergency response organization as a result of confirming the worker's location.

In step S2050, the knowledge service providing server 200 guides general workers other than the emergency response organization to the evacuation location as a result of confirming the worker location.

In step S2055, the wearable device 300 of the worker of the emergency response organization confirms the emergency response command.

In steps S2060 to S2065, the wearable device 300 of a worker of an emergency response organization checks emergency response information, determines the location of emergency response chemicals or equipment, and performs emergency response using this.

In step S2070, the knowledge service providing server 200 receives an emergency response result from the wearable device 300 of a worker of an emergency response organization. When evacuation is necessary, the knowledge service providing server 200 can guide the wearable device 300 of workers of an emergency response organization to an evacuation location.

Referring to FIG. 21, when a sudden leak in a pipe requiring emergency response occurs, the knowledge service providing server 200 provides content on the layout of the control equipment and how to use it to the wearable device 300 of the worker of the emergency response organization. You can perform emergency measures and receive a report on the completion of emergency measures after taking emergency measures.

Methods for providing knowledge services according to various embodiments of the present invention may be implemented in the form of program instructions that can be executed through computer means such as various servers. Additionally, programs and applications for executing the method for providing knowledge services according to the present invention may be installed on a computer and recorded on a computer-readable medium. Computer-readable media may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on a computer-readable medium may be specially designed and constructed for the present invention or may be known and usable by those skilled in the computer software field. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes magneto-optical media and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc.

So far, the present invention has been examined focusing on its embodiments. A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative rather than a restrictive perspective. The scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope should be construed as being included in the present invention.

100: Augmented reality authoring terminal
200: Knowledge service provision server
300: Wearable device (smart glasses)

Claims (6)

delete In the knowledge service provision method provided by the knowledge service provision server,
The knowledge service providing server,
It includes providing augmented reality content created by linking objects, markers, and information from an augmented reality content authoring tool through communication with a wearable device and supporting the activities of the wearable device,
The method by which the knowledge service providing server provides the knowledge service is,
a) detecting the occurrence of an event;
b) confirming the occurrence area of the detected event through a sensor or camera connected through wired or wireless communication;
c) If the type of event requires emergency maintenance of a facility or equipment, checking the location of the emergency maintenance worker by referring to the attendance record information and work schedule information database;
d) selecting the worker to be assigned to emergency repair by selecting the worker located closest to the emergency repair facility or equipment based on the location confirmation result;
e) instructing the selected worker to provide emergency maintenance and instructing the next-highest priority worker to provide emergency maintenance support; and
f) A method of providing knowledge services, including receiving a report of completion of emergency repairs. delete delete According to paragraph 2,
In step e) above,
e-1) Recognizing a facility or equipment subject to emergency maintenance by the wearable device of the selected worker;
e-2) outputting emergency maintenance content corresponding to the facility or equipment subject to emergency maintenance by the selected worker's wearable device; and
e-3) A knowledge service providing method further comprising transmitting an emergency repair completion report to the knowledge service providing server after emergency repair is performed based on the content by the selected worker's wearable device. A computer-readable recording medium on which a computer program executing the knowledge service provision method of paragraph 2 or 5 is recorded.

Images