KR20130011037A - Knowledge based augmented reality system - Google Patents

Knowledge based augmented reality system Download PDF

Info

Publication number
KR20130011037A
KR20130011037A KR1020110071912A KR20110071912A KR20130011037A KR 20130011037 A KR20130011037 A KR 20130011037A KR 1020110071912 A KR1020110071912 A KR 1020110071912A KR 20110071912 A KR20110071912 A KR 20110071912A KR 20130011037 A KR20130011037 A KR 20130011037A
Authority
KR
South Korea
Prior art keywords
information
augmented reality
virtual object
knowledge
situation
Prior art date
Application number
KR1020110071912A
Other languages
Korean (ko)
Inventor
김상철
우종우
최진우
신기현
Original Assignee
국민대학교산학협력단
건국대학교 산학협력단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 국민대학교산학협력단, 건국대학교 산학협력단 filed Critical 국민대학교산학협력단
Priority to KR1020110071912A priority Critical patent/KR20130011037A/en
Publication of KR20130011037A publication Critical patent/KR20130011037A/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Tourism & Hospitality (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • General Business, Economics & Management (AREA)
  • Strategic Management (AREA)
  • Primary Health Care (AREA)
  • Marketing (AREA)
  • Human Resources & Organizations (AREA)
  • General Health & Medical Sciences (AREA)
  • Economics (AREA)
  • Computer Hardware Design (AREA)
  • General Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Geometry (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The present invention provides an augmented reality (AR) system.
Augmented reality system of the present invention, the situation recognition layer for receiving the information of the real world from the outside to generate the situation information of the real object existing in the real world and recognize the generated situation information; An information processing layer that infers the operation of the virtual object and the virtual object by a predetermined inference rule according to the situation information of the real object transmitted from the situation recognition layer; An information augmentation layer that generates the virtual object to be augmented to the real object based on the inference result transmitted from the information processing layer and matches the generated virtual object with the image of the real world to implement augmented reality; And an input / output unit configured to receive information input from a user and display an augmented reality implemented in the information enhancement layer. .

Description

Knowledge-based Augmented Reality System {KNOWLEDGE BASED AUGMENTED REALITY SYSTEM}

The present invention relates to an augmented reality (AR) system, and in particular, in an environment in which virtual objects are synthesized in a space of the real world, the virtual objects can operate intelligently by reflecting contextual information and enable interaction between multiple objects. It relates to a knowledge-based augmented reality system.

Recently, augmented reality (AR) is becoming more and more popular due to the spread of various information devices mixed with a camera and a display. Augmented reality is an environment in which virtual objects are synthesized in the space of the real world, substantiating information about virtual situations and matching them to images to assist human cognitive ability through visualization of information.

In the early stages of research, the main focus was on how to construct augmented reality environment and registration problems to match real objects and virtual objects.In recent years, the research was conducted to convey knowledge about information that users cannot actually feel. Beyond function, research is being conducted on various applications such as education, design, tourism, and industrial fields.

In general, augmented reality system can be divided into tracking, virtual object generation, image registration. Tracking is to recognize the state of the object in the real world to augment the information, for example, a technique that utilizes a square marker, a color marker, a feature point, and the like has been studied. Virtual object generation is a step of generating a 3D model according to the state of the tracked object, such as VRML OpenGL. In image matching, an augmented reality environment is mixed with reality and virtual objects through the synthesis of input and rendered images.

Representative conventional augmented reality research results are as follows.

ArToolKit, ARVIKA, Auto-assembly, Outdoor tracking, CyliCon, ArLoc and CyberCode using square markers are presented. In particular, in the case of ARVIKA system based on image recognition technology, the user matches the existing object using the method of augmenting the 3D model corresponding to the ID, position, and direction of the tracked object. Do the work. These methods find a specific pattern in the image, recognize the location and direction of the pattern, and match the 3D model based on it.

In addition, a method of tracking a user's position and direction by using a GPS and a magnetometer and augmenting and using peripheral geographic data is also used. Representatives include Exploring Mars and Underground Infrastructure Visualization.

In addition, the AR maintenance tool animate 3D models based on maintenance process scenarios to induce the operator to operate according to the animation.In Manufacturing Planning, the 3D model of the worker's performance radius can be used to design a manufacturing plant. Make sure

However, such a conventional augmented reality system has a problem in the basic technology, such as limited computing resources, the size and weight of the display device, high cost, limited power supply, depending on the application purpose. However, the following limitations exist because the purpose of development is mainly based on 3D models rather than hardware problems due to the mobility of equipment.

First, in operating the augmented 3D model, there is a problem in that it cannot reflect various real-world situations, and performs only a predetermined operation sequentially according to a user's command input. In addition, since most systems store and manage markers and 3D models individually, a 3D model corresponding to the tracked information 1: 1 must be produced for each situation. In addition, the augmented object is stored based on the 3D model, and since the 3D model describes only three-dimensional information, there is a problem in that it does not represent the state of various objects.

The present invention has been proposed to solve the above problems of the prior art, in which the virtual object reflects the situation information in an environment where the virtual object is synthesized in the real world space, and intelligently operates to enable interaction between multiple objects. The purpose is to provide a knowledge-based augmented reality system.

In addition, the present invention provides a knowledge-based augmented reality system that can infer a variety of data by applying a situation-aware agent based on the situation or system using a knowledge-based process to intelligently reflect the current situation in the system. There is an additional purpose.

Knowledge-based augmented reality system of the present invention for achieving the above object,

A situation cognition layer that receives information of the real world from the outside, generates situation information of a real object existing in the real world, and recognizes the generated situation information; An information processing layer that infers the operation of the virtual object and the virtual object by a predetermined inference rule according to the situation information of the real object transmitted from the situation recognition layer; An information augmentation layer that generates the virtual object to be augmented to the real object based on the inference result transmitted from the information processing layer and matches the generated virtual object with the image of the real world to implement augmented reality; And an input / output unit configured to receive information input from a user and display an augmented reality implemented in the information enhancement layer. It includes.

In an embodiment of the present invention, the virtual object is preferably a 3D model.

In an embodiment of the present disclosure, the situational awareness layer may include: a sensing unit configured to acquire information of the real world as input data; And a situation information generation unit generating situation information of the real object by using the acquired input data. It includes.

Here, the situational awareness layer, tracking the square marker in the image image input through the sensing unit, infer the information of the real world based on the current position and the direction information of the tracked marker and the inferred result Further comprising a first agent for delivering to the processing layer.

The situational awareness layer may track a click event of a specific coordinate (X, Y) by a user's input in the augmented reality viewer displayed on the input / output unit and process the information on the tracked click event information. It further comprises a second agent for delivering to the layer.

In an embodiment of the present disclosure, the information processing layer may include: a knowledge base storing base knowledge on an operation of the virtual object that is augmented according to an object of an application program of the knowledge-based augmented reality system; And an inference engine extracting an operation of an augmented virtual object by applying a predetermined inference rule based on the base knowledge stored in the knowledge base and the situation information input from the outside.

Here, the inference engine infers the operation of the virtual object using the relationship between the plurality of virtual objects or between the virtual object and the real object.

In an embodiment of the present disclosure, the information augmentation layer may include: a virtual object generation unit configured to generate the virtual object by analyzing information on the operation of the virtual object transmitted from the information processing layer; And an image editing unit for matching the generated virtual object with the real object of the real world and visualizing the same. It includes.

Knowledge-based augmented reality system of the present invention has the following effects.

First, since it is possible to infer a variety of data through the situation-aware agent using the knowledge-based process, it is possible to intelligently reflect the current situation information about the virtual object in the augmented reality system.

Second, by abstracting the virtual objects existing in the augmented reality system into a single class, various virtual models can be created.

Third, it is possible to implement the dynamic function and flow of execution of virtual objects according to the current occurrence of the knowledge-based process.

Fourth, it is possible to facilitate the developer's convenience through the framework for developing the augmented reality system of the present invention.

1 is a block diagram of a knowledge-based augmented reality system according to an embodiment of the present invention.
2 is a block diagram of a situational awareness layer according to the present invention.
3 is a view for explaining a process of generating situation information of a first situation awareness agent according to an embodiment of the present invention.
4 is a block diagram of an information processing layer according to the present invention.
5 is an exemplary view of a knowledge base showing a procedure of performing a block assembly model in an augmented reality system according to the present invention.
6 is an exemplary view of the inference tree of the inference engine according to an embodiment of the present invention.
7 is a block diagram of an information augmentation layer according to the present invention.
8 is a view for explaining the role of the object control unit according to the present invention.
9 is a block diagram for explaining the operation of the knowledge-based augmented reality system according to an embodiment of the present invention.
10 is a view showing a verification example of the knowledge-based augmented reality system of the present invention.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a block diagram of a knowledge-based augmented reality system according to an embodiment of the present invention.

Referring to FIG. 1, the knowledge-based augmented reality system 100 according to the present invention includes a context awareness layer 110, an information processing layer 120, an information augmentation layer 130, and an input / output unit 140. It is composed. This knowledge-based augmented reality system 110 is composed of three layers (110, 120, 130) as shown in the drawing in consideration of the recognition and utilization of various contextual information, dynamic generation of 3D models, sharing and reusability of the model is implemented from there Augmented reality is finally output through the input / output unit 140.

The context awareness layer 110 processes various external data for context awareness as the lowermost layer of the system 100. To this end, by processing the information of the real world input from various sensors to generate the situation information of the real object in the real world and recognizes the generated situation information.

The information processing layer 120 utilizes the contextual information of the real object transmitted from the contextual awareness layer 110 to detect the virtual object and the operation of the virtual object that should be augmented to the real object by a predetermined inference rule. Infer.

The information augmentation layer 130 generates a virtual object to be augmented to a real object based on the inference result transmitted from the information processing layer 120 and matches the generated virtual object with an image of the real world. Implement augmented reality

The input / output unit 140 receives various information input by the user and displays augmented reality implemented in the information augmentation layer 130.

2 is a block diagram of a situational awareness layer according to the present invention.

Referring to FIG. 2, the contextual awareness layer 110 according to the present invention performs a function of inferring contextual information through inference through the state of a real object existing in an augmented reality environment, and obtained through the inference result. The information is transmitted to the information processing layer 120 which is a higher layer. For this purpose, the contextual recognition layer 110 includes a sensing unit 111, a context information generating unit 112, and a plurality of contextual awareness agents 113.

The sensing unit 111 acquires information of the real world as input data. Real-world information includes real object information, which includes the information needed to augment virtual objects to real objects. Such real-world information is input through various sensors.

The situation information generator 112 generates situation information of an actual object by using input data acquired through the sensing unit 111.

At least one situation-aware agent 113 may be included. In this situation, the agent 113 performs individual object tracking or receives additional tracking results to generate situation information according to each performance purpose. An embodiment of the present invention includes two context aware agents.

For example, the first agent is a Mark Tracker Agent, which tracks a square marker in an image image input through the sensing unit 111 and displays information of the real world based on the current position and direction information of the tracked marker. Inferring and transfers the inferred result to the information processing layer 120. The second agent is a click event handler agent, which tracks and tracks click events of specific coordinates (X, Y) by user input in the augmented reality viewer displayed on the input / output unit 140. Transferred information to the information processing layer 120. In particular, when a user performs interaction A by clicking a specific coordinate (X, Y) of the augmented reality viewer, the click event handler agent generates a message interactionA (X, Y) and delivers it to the information processing layer 120. .

3 is a view for explaining a process of generating situation information of a first situation awareness agent according to an embodiment of the present invention.

As shown in FIG. 3, the marker tracking agent as the first situational awareness agent according to an embodiment of the present invention tracks a square marker in an input image image and infers and generates contextual information of the real world. In the present invention, for the purpose of realizing an augmented reality environment reflecting the situation information of the real world through various situation recognition agents, the relationship between the blind spots is regarded as a kind of situation information, and the actual markers and detection algorithms are widely provided by ARToolKt. Basic square markers and marker detection algorithms are used. After tracking the marker, the marker tracker agent additionally generates the inferred relation between the markers and the situation in the real world based on the tracked marker position and direction information, and then transfers the current state of the augmented reality object to the upper layer 120. To pass.

4 is a configuration diagram of an information processing layer according to the present invention.

Referring to FIG. 4, the information processing layer 120 according to the present invention determines how a virtual object model should operate in order to dynamically generate a virtual object model according to a situation recognized by the situation recognition layer 110. Inferring, and transfers the result of the inference to the information enhancement layer 130, which is a higher layer. To this end, the information processing layer 120 includes a knowledge base 121 and an inference engine 122.

The knowledge base 121 stores base knowledge of how each virtual object to be augmented in the application operates according to the purpose of the application implemented through the framework.

The inference engine 122 infers the operation of the virtual object model of the augmented reality that should be augmented in the current state of the real world by applying a predetermined reasoning rule based on the knowledge base 121 and the contextual information input from the outside. Transfer to the information augmentation layer 130. Details of the information presentation method and the inference process of the information processing layer 120 are as follows.

In general, the knowledge base 121 means that data about a specialized area of an application program to be implemented is represented by a knowledge-based expression technique. Applying the concept of the domain knowledge of artificial intelligence to the augmented reality system can describe the state of the virtual objects of the augmented reality and the future performance, which can be expressed as shown in the knowledge base of FIG.

5 is an example of a knowledge base showing a procedure of performing a block assembly model in an augmented reality system according to the present invention.

Referring to FIG. 5, in order to perform block assembly of an augmented reality system, "definition of an augmented reality virtual object model", "a rule of situation awareness agent", and "a marker tracking rule of situation awareness agent" are defined. In the example of FIG. 5, the BlockA model, which is an instance of the ARModelModel of augmented reality, has two assembly points left (LeftPart) and right (RightPart). In addition, it represents whether the block state is a join state or a disjoin state, and defines it as a function such as assembly and disassembly. In addition, the knowledge base has an inference rule for the situation-aware agent to infer the current situation. For example, rule 3 infers that "If BlockA, BlockB exists and the left part of the blocks is not combined, combine it". May have rules In addition to the inference rules for this assembly, the context aware agent additionally includes inference rules for tracking the marker by recognizing the current situation.

6 is an exemplary view of an inference tree of an inference engine according to an embodiment of the present invention.

The knowledge base 121 of FIG. 5 described above has a tree structure as shown in FIG. 6 based on the AND node concept of the AND / OR tree. Each node represents an inferred result, and the child nodes connected to the parent node represent one rule. If the state of the lower node is satisfied, the motion of the 3D model corresponding to the upper node is augmented. The inference engine 122 infers the current position in the tree structure with the situation transmitted from the agent, and delivers the result of the inference to the information augmentation layer 130 to dynamically create the virtual object.

7 is a configuration diagram of an information enhancement layer according to the present invention.

Referring to FIG. 7, the information augmentation layer 130 according to the present invention generates a 3D model of a virtual object to be augmented based on the inference result transmitted from the information processing layer 120, which is a lower layer, and displays an image of the real world. The generated 3D model is matched to form a final augmented reality scene to be delivered to the input / output unit 140 to be displayed to the user. To this end, the information augmentation layer 130 includes an object controller 131 including an virtual object generator 130a and an image editor 132.

The virtual object generator 131a analyzes information on the operation of the virtual object transmitted from the information processing layer 120 to generate a virtual object, and the image editor 132 generates the virtual object and the real object in the real world. Match and visualize it.

On the other hand, the object controller 131 analyzes the information on the operation of the augmented reality virtual object transmitted from the lower layer to generate a 3D model and match the existing image through the image editing unit 132 to provide to the user. At this time, the received information describes the conceptual behavior of the augmented reality object model, not the data for directly implementing the 3D model, and the role of actually generating the 3D model based on the virtual model generator (not shown), The image editing unit 132 performs a matching and visualization role of the generated 3D model. Also, the information shared by the object controller 131 through the lower layer and the data shared through the virtual object model of the augmented reality are shown in FIG. 8.

8 is a view for explaining the role of the object control unit according to the present invention.

In the example shown in FIG. 8, "Behavior" means an action to be performed as shown in FIG. 6, "Related Model" means the object of the action, and "information to be augmented" is detailed information related thereto. For example, "BlockA->Mesh->LeftPart" means assembly of LeftPart at the coordinate where BlockA is to be displayed.

9 is a block diagram illustrating an operation of a knowledge-based augmented reality system according to an embodiment of the present invention.

Referring to FIG. 9, in the knowledge-based augmented reality system according to the present invention, a computer or the like is matched with an image of a real world based on a coordinate and a direction of a marker input from an image through a camera. Provide information visually to people. In particular, the information processing layer 120 and the situation recognition layer 110 in the framework according to the present invention on such an augmented reality environment is a concept of extending the main process of the existing system. Existing augmented reality systems show a static performance flow for one main process, while the augmented reality system of the present invention receives the current situation from the situation recognition agent of the situation recognition layer 110, the knowledge base 121 of the target area. According to the situation inferred through), the appropriate virtual object is created and provided to the user. Therefore, it is possible to transmit information more dynamically by applying various augmented objects according to the change of the current object state or other environment.

Referring to FIG. 9, the external data acquired by the plurality of sensing units 111 is input to the situation recognition agent 113, and the situation recognition agent 113 performs individual object tracking to perform situation information of the real world from the external data. Check it. The inference engine 122 infers the operation of the virtual object that is augmented by applying a predetermined inference rule to the information transmitted from the knowledge base 121 and the situation awareness agent 113. The result inferred as described above is input to the virtual object generator 131a. The virtual object generating unit 131a analyzes the inferred result to generate a 3D model of the virtual object, and the image editing unit 132 finally synthesizes the 3D model of the virtual object into the image of the real object. ) To output augmented reality (AR) image. In this case, the virtual object generator 131a uses the virtual object model 150 to generate a 3D model of the virtual object. Hereinafter, the virtual object model 150 will be described in detail.

In the knowledge-based augmented reality system according to the present invention, the context-sensitive agent 113, the inference engine 122, the object controller 131, which are representative components of each layer 110, 120, and 130 are the most basic elements that are the foundation of the augmentation. The virtual object model 150 is shared. The virtual object model 150 depicts the characteristics of the real object in the real world that is augmented. For example, the ARObjectModel class of the augmented reality virtual object model depicts the location, 3D information, and structural features of the real object using MARKER, MESH, and SUB-PART as member variables, as shown in the following example of the augmented reality virtual object model. BlockAModel is an instance of ARObjectModel that stores the location of the data in each member variable. It also has a subpart, called SubPartA, which can be in a combined or unjoined state and handles the assembly and disassembly of the part.

The virtual object model 150 divides an existing unit object into smaller parts that can be interacted with, and hierarchically depicts a set of these units, each part of the target object, the state of each part, It contains a list of actions that can be performed and three-dimensional information. That is, by classifying and sharing augmented reality objects, each layer may share one meaning and perform functions of each layer based on the same. The situation-aware agent 113 describes the tracking result from the sensor data based on the state of the virtual object model 150, and the inference engine 122 recognizes the situation through this and infers the inference based on the knowledge base 121. The object controller 131 dynamically generates a 3D model by referring to the virtual object model 150 and synthesizes the behavior of the virtual object model input from the inference engine 122 and combines the shape of the real object in the real world.

Details of each of the layers 110, 120, and 130 and their representative components are as follows.

Figure pat00001

On the other hand, the present invention designs a knowledge-based augmented reality framework for the construction of a more convenient augmented reality system. This framework constitutes the central part of the components (components) according to the augmented reality system structure described above, and easily establishes a knowledge-based augmented reality environment as the augmented reality developer defines a situation model, a knowledge base, and an augmented reality virtual object model. Make it configurable. The framework of the present invention consists of a mastication unit, a driving unit and a monitoring unit as shown in Table 1 below.

Copyright Driving part Monitoring Project integration management Project DB Project Navigator Augmented Reality Virtual Object Model Performance engine Message control Knowledge base Inference Engine Viewer Augmented Reality Viewer

The authoring department includes functions used to implement a knowledge-based augmented reality environment, and includes a rule editor, an augmented reality virtual object editor, and a marker editor. Developers can edit and save the augmented reality virtual object model and context awareness agent's context model and knowledge base through this editor. The driving unit includes functions related to program execution and includes a knowledge base, inference engine, augmentation control unit, and situational awareness agent for loading and inferring augmented reality models. The monitoring unit provides a view to monitor the status, current situation, and inference of each model when the knowledge-based augmented reality environment is driven.

The framework of the present invention supports the following functions according to considerations in designing knowledge-based augmented reality systems.

First, in order to recognize and utilize various context information, the process is performed by receiving the current situation from the context agent. This enables the application programs implemented through the framework to reflect the current situation in the process, and can implement a more intelligent augmented reality system as the process flow is not fixed but dynamically performed.

Second, by describing each object appearing in augmented reality through an augmented reality object model, it is possible to dynamically generate and augment a 3D model based on the position of each sub-part of the object model. While the existing augmented reality system stores the marker and the 3D model separately and statically augments the 3D model according to the detected markers, the augmented reality object model infers the information about each object to identify the relationship between the objects and based on them. You can dynamically generate 3D models.

Third, the framework stores and manages the domain knowledge and augmented reality object model separately from the main process to enable sharing and reuse of existing implementations when implementing new augmented reality programs.

[Experimental Example]

The knowledge-based augmented reality system of the present invention uses a webcam in the Windows environment, and implemented based on the library of NyARToolKit and Eclipse.

In addition, the implementation of each module of the three parts of the augmented reality system (operation part, driving part, monitoring unit) is described for each module in the following, the entire framework as a demonstration system by selecting the equipment maintenance area as shown in FIG. Verified.

First, the authoring function supports the authoring of augmented reality object model and knowledge of a specific field, and in detail, it is divided into an object model editor and a knowledge editor. The object model editor can register the object's name, three-dimensional data, and rectangular pattern, and can register the lower part and the state of the lower part of the object. The knowledge editor can save and edit the expertise of the subject area in the form of rules with IF / THEN clauses. Once a rule is complete, the newPhrase button will allow you to add new rules sequentially. The authored augmented reality object model and specific field knowledge are stored in project data and can be filed in and out.

Next, the driving unit is composed of an inference engine, an AR viewer and an marker tracking agent, and is performed when the authored model is driven. The inference engine is performed in the system, and the marker tracking agent shows the user the ID of the tracked marker to the user, and finally the augmented reality viewer outputs the image.

Finally, the monitoring unit consists of a Project Navigator and a window for viewing the parameters inside the engine. Project Navigator displays the current working status by displaying the model, situational agent, and knowledge included in the current working project in tree form. The window displays the Production Memory, Working Memory, 3D Model List, Augmented Reality Object List, and Marker Lists that are performed on the engine of the running project to monitor the status inside the engine.

[Experiment result]

In order to test the knowledge-based augmented reality system proposed in the present invention, a printing device for inspecting RFID media was selected as a target area. Although the operation of the mechanical device is manual-oriented, the augmented reality technique can be an effective education and training for the user, so it was selected as an experiment area. The situation was recognized by collecting the events occurring in the RFID production system by the agent, and the user's actions were induced by augmenting the measures according to the recognized situation.

10 is a view showing a verification example of the knowledge-based augmented reality system of the present invention.

As shown in FIG. 10, the virtual model depicting a control unit such as a button is displayed in blue, and a portion that can be manipulated by a user is displayed in light blue, as can be seen from the result shown in the verification example. In addition, the action that the user should take now is augmented with red color to induce the user's action. Since the result of the RFID tag test cannot be visually identified, the communication test is performed to visualize the decision result and then augment it to the corresponding tag to enable rapid processing in the event of a defect.

As described above, in the present invention, as a result of the trial implementation using the proposed framework, the situation that occurs in the experimental environment according to the pre-stored knowledge base without the user's command in the knowledge-based augmented reality system can be augmented to the user. In addition, it was possible to implement a system that is interactive with the external environment than the existing augmented reality system. The present invention proposes a knowledge-based augmented reality system applying a knowledge-based process to an augmented reality system. To this end, we used a situation awareness agent to recognize various situations and augmented a dynamic virtual model through an inference engine and an augmentation controller. Also, by making this framework, the virtual object model can be shared and reused, and the augmented reality environment can be easily configured according to the proposed system structure.

The importance of the present invention is as follows. First, it can recognize and utilize various situation information through situational awareness agent. Second, it can perform more interactive and intelligent execution flow according to situation by recognizing and reflecting it according to user's input or status. Third, by abstracting and describing augmented reality objects, it is possible to realize a more flexible and object-oriented augmented reality environment. Fourth, it is possible to share and reuse the model implemented through the framework, and to implement augmented reality environment based on the structure of the system to be easily developed.

In addition, the framework proposed in the present invention does not consider a function for implementing a context aware agent, and configures the framework on the premise that the context aware agent operates correctly according to the context information list required by the framework. Therefore, for a more realistic knowledge-based augmented reality framework, further research on logical errors and validation of authored augmented reality virtual object models, knowledge bases, and context-aware agents should be continued.

While the present invention has been described in detail through the preferred embodiments, it should be understood that the present invention is not limited to the contents of these embodiments. Those skilled in the art will appreciate that various modifications, additions and substitutions are possible, without departing from the scope of the appended claims, The genius will be so self-evident. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Recently, due to the spread of various information devices equipped with cameras and display devices, augmented reality technology has been applied to education, design, tourism, and industrial fields. In the case of such augmented reality technology, research to reflect a wide range of user's situations is continuously being conducted. In particular, the augmented reality system has been expanded to a more diverse and personalized service such as attempting to integrate with a situation recognition system.

In view of this, the present invention can intelligently reflect the current situation to the system by inferring various data through the situation awareness agent, which is very useful not only for the information equipment industry but also for industrial fields closely related to real life such as design, education and tourism. May be applied.

110: situational awareness layer 111: sensing unit
112: situation information generation unit 113: situation awareness agent
120: information processing layer 121: knowledge base
122: reasoning engine 130: information enhancement layer
131: augmentation control unit 131a: virtual object generation unit
132: image editing unit 140: input / output unit

Claims (8)

A situation cognition layer that receives information of the real world from the outside, generates situation information of a real object existing in the real world, and recognizes the generated situation information;
An information processing layer that infers the operation of the virtual object and the virtual object by a predetermined inference rule according to the situation information of the real object transmitted from the situation recognition layer;
An information augmentation layer that generates the virtual object to be augmented to the real object based on the inference result transmitted from the information processing layer and matches the generated virtual object with the image of the real world to implement augmented reality; And
An input / output unit configured to receive information input from a user and display an augmented reality implemented in the information enhancement layer; Knowledge-based augmented reality system comprising a. The method of claim 1,
The knowledge-based augmented reality system, characterized in that the virtual object is a 3D model. The method of claim 1, wherein the situation awareness layer,
A sensing unit that acquires the information of the real world as input data; And
A situation information generation unit generating situation information of the real object by using the acquired input data; Knowledge-based augmented reality system comprising a. The method of claim 3, wherein the situation awareness layer,
A first agent that tracks a square marker in the image image input through the sensing unit, infers the information of the real world based on the current position and direction information of the tracked marker, and transfers the inferred result to the information processing layer. Knowledge-based augmented reality system characterized in that it further comprises. The method of claim 4, wherein the situation awareness layer,
And a second agent that tracks a click event of a specific coordinate (X, Y) by a user's input in the augmented reality viewer displayed on the input / output unit and delivers the tracked click event information to the information processing layer. Knowledge-based augmented reality system, characterized in that. The method of claim 1, wherein the information processing layer,
A knowledge base storing base knowledge on the operation of the virtual object, which is augmented according to the purpose of the application program of the knowledge-based augmented reality system; And
Knowledge base augmented reality system comprising a reasoning engine for extracting the operation of the augmented virtual object by applying a predetermined inference rule based on the base knowledge stored in the knowledge base and the situation information input from the outside. The method of claim 6, wherein the inference engine,
The knowledge-based augmented reality system, characterized in that the inference of the operation of the virtual object using the relationship between the plurality of virtual objects, or between the virtual object and the real object. The method of claim 1, wherein the information enhancement layer,
A virtual object generation unit for generating the virtual object by analyzing information on the operation of the virtual object transmitted from the information processing layer; And
An image editor for matching the generated virtual object with the real object in the real world and visualizing the same; Knowledge-based augmented reality system comprising a.
KR1020110071912A 2011-07-20 2011-07-20 Knowledge based augmented reality system KR20130011037A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020110071912A KR20130011037A (en) 2011-07-20 2011-07-20 Knowledge based augmented reality system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020110071912A KR20130011037A (en) 2011-07-20 2011-07-20 Knowledge based augmented reality system

Publications (1)

Publication Number Publication Date
KR20130011037A true KR20130011037A (en) 2013-01-30

Family

ID=47840078

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020110071912A KR20130011037A (en) 2011-07-20 2011-07-20 Knowledge based augmented reality system

Country Status (1)

Country Link
KR (1) KR20130011037A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101360999B1 (en) * 2013-08-09 2014-02-10 코리아디지탈 주식회사 Real time data providing method and system based on augmented reality and portable terminal using the same
WO2019117655A1 (en) * 2017-12-15 2019-06-20 네이버랩스 주식회사 Remote location-based ar authoring method and system using 3d map
KR102402170B1 (en) * 2021-10-22 2022-05-26 주식회사 제이어스 Apparatus and method for generating metaverse using image analysis

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101360999B1 (en) * 2013-08-09 2014-02-10 코리아디지탈 주식회사 Real time data providing method and system based on augmented reality and portable terminal using the same
WO2019117655A1 (en) * 2017-12-15 2019-06-20 네이버랩스 주식회사 Remote location-based ar authoring method and system using 3d map
KR102402170B1 (en) * 2021-10-22 2022-05-26 주식회사 제이어스 Apparatus and method for generating metaverse using image analysis
WO2023068795A1 (en) * 2021-10-22 2023-04-27 주식회사 제이어스 Device and method for creating metaverse using image analysis

Similar Documents

Publication Publication Date Title
Devagiri et al. Augmented Reality and Artificial Intelligence in industry: Trends, tools, and future challenges
Delgado et al. A research agenda for augmented and virtual reality in architecture, engineering and construction
Palmarini et al. A systematic review of augmented reality applications in maintenance
Gimeno et al. A new AR authoring tool using depth maps for industrial procedures
Ong et al. A novel maintenance system for equipment serviceability improvement
Hajirasouli et al. Augmented reality in design and construction: Thematic analysis and conceptual frameworks
WO2019217159A1 (en) Immersive feedback loop for improving ai
Barakonyi et al. Ubiquitous animated agents for augmented reality
Abate et al. An interactive virtual guide for the AR based visit of archaeological sites
Behzadan ARVISCOPE: Georeferenced visualization of dynamic construction processes in three-dimensional outdoor augmented reality
US11928384B2 (en) Systems and methods for virtual and augmented reality
KR20130011037A (en) Knowledge based augmented reality system
Ssin et al. Geovcm: Virtual urban digital twin system augmenting virtual and real geo-spacial data
Ivaschenko et al. Accented visualization for augmented reality
Behzadan et al. Enabling smooth and scalable dynamic 3D visualization of discrete-event construction simulations in outdoor augmented reality
CN113989462A (en) Railway signal indoor equipment maintenance system based on augmented reality
LaViola Jr et al. Analyzing slam algorithm performance for tracking in augmented reality systems
Rampini et al. Synthetic images generation for semantic understanding in facility management
Dias et al. A4D: augmented reality 4D system for architecture and building construction
Hempe et al. A semantics-based, active render framework to realize complex eRobotics applications with realistic virtual testing environments
Hung et al. Service-based simulator for security robot
Khakpour et al. What We Know About the Greenability of Reality Technologies: A Systematic Literature Review
Tran Framework for artificial intelligence assisted augmented reality systems for education and training
Wang [Retracted] Application of VR Virtual Reality in the Course of Later Stage and Special Effect Production
Bekaroo et al. Ai-assisted extended reality toward the 6g era: challenges and prospective solutions

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E601 Decision to refuse application