KR20220107362A - System and method for providing augmented reality service - Google Patents

Abstract

The present invention relates to a system for providing an augmented reality service. The system comprises: a communication server for mediating remote collaboration between terminals located in a remote location; a first terminal for capturing images of a real world and transmitting the images to the communication server in real time; a second terminal for generating augmented reality content for the image of the real world received through the communication server and transmitting the augmented reality content to the first terminal in real time through the communication server; and a storage server for storing remote collaboration data including remote collaboration content based on the image of the real world and the augmented reality content received through the communication server.

Description

Augmented reality service providing system and method {System and method for providing augmented reality service}

The present invention relates to a system and method for providing an augmented reality service, and more particularly, to a system and method for storing and providing remote collaboration contents as data while performing remote collaboration using augmented reality.

Recently, with the rapid development of 3D image processing technology, virtual environment services using virtual reality (VR) or augmented reality (AR) are being provided in various fields such as movies, games, and interiors. Among them, augmented reality is a technology for providing a user with an image without a boundary between the real world and the virtual world by synthesizing and displaying a real image and virtual augmented reality content.

These virtual environment services are being used in various fields such as remote medical diagnosis, architectural design, and manufacturing process management. For example, when a user takes a picture of a medical device with a camera, augmented reality content corresponding to the corresponding medical device may be synthesized and displayed on the medical device image. These virtual environment services enable users to use medical equipment correctly and appropriately.

Meanwhile, recently, a service (hereinafter referred to as a remote collaboration service) that enables remote users to share and collaborate on virtual environment images in real time through an online platform is being developed.

The present invention provides a system and method for providing an augmented reality service that enables remote users to share an augmented reality image in real time through an augmented reality service and to collaborate remotely.

In addition, the present invention provides an augmented reality service providing system and method for storing and managing remote collaboration contents when performing remote collaboration using augmented reality.

Augmented reality service providing system according to an embodiment for solving the above-described problems, a communication server that mediates remote collaboration between terminals located in remote places, a first image of the real world is captured and transmitted in real time to the communication server A terminal, a second terminal that generates augmented reality content for the image of the real world received through the communication server and transmits it in real time to the first terminal through the communication server, the real world received through the communication server On the basis of the image and the augmented reality content, including a storage server for storing remote collaboration data including the content of the remote collaboration, wherein the first terminal is augmented by synthesizing the augmented reality content with the image of the real world It can be displayed as a real image.

The storage server may generate one image by synthesizing the augmented reality content with the image of the real world, and store the generated image as the remote collaboration data.

The storage server may store the image of the real world and the augmented reality content separately in separate storages.

The augmented reality content may include a graphic object and a file object, and the storage server may store the graphic object and the file object separately in separate storages.

The storage server may generate mapping information including time information and location information on the image of the real world in which the augmented reality content is to be synthesized, and may further store it as the remote collaboration data.

The time information may be set as a reproduction time interval value or a frame value for the image of the real world.

The location information may be set as a coordinate value for an arbitrary reference point in the image of the real world or information on an anchor identified in the image of the real world.

The anchor may be any object identified through image analysis in the image of the real world.

The storage server may transmit the remote collaboration data to any terminal including the first terminal and the second terminal.

The arbitrary terminal displays the augmented reality image by synthesizing the augmented reality content included in the remote collaboration data using mapping information to the real world image included in the remote collaboration data, thereby sharing the content of the remote collaboration can do.

The image of the real world may include an image of at least one medical device, and the augmented reality content may include information for guiding a usage of the at least one medical device.

A method of providing an augmented reality service according to an embodiment includes the steps of: by a first terminal, capturing an image of the real world and transmitting the image to a communication server; Transmitting an image of, the second terminal generating the augmented reality content for the image of the real world and transmitting it to the communication server, the communication server transmitting the augmented reality content to the first terminal , the step of the first terminal synthesizing the augmented reality content with the image of the real world and displaying it as an augmented reality image, the communication server transmitting the image of the real world and the augmented reality content to the storage server to the storage server and storing, by the storage server, the remote collaboration data based on the image of the real world and the augmented reality content.

The storing of the remote collaboration data may include generating one image by synthesizing the augmented reality content with the image of the real world, and storing the generated image as the remote collaboration data.

The storing of the remote collaboration data may include generating a graphic object or a file object according to the type of the augmented reality content, and separately storing the image of the real world, the graphic object, and the file object in a separate storage. may include steps.

The storing of the remote collaboration data may include generating mapping information including time information and location information on the image of the real world to which the augmented reality content is to be synthesized, and matching the mapping information to the image of the real world. can be stored in a separate storage.

The time information may be set as a reproduction time interval value or a frame value for the image of the real world.

The generating of the mapping information may include: identifying an anchor in the image of the real world with respect to a position where the augmented reality content is to be synthesized in the image of the real world; and converting the information about the anchor into the position information. It may include a setting step.

The method may further include, by the storage server, transmitting the remote collaboration data to any terminal including the first terminal and the second terminal.

The method may further include the step of displaying an augmented reality image by synthesizing the augmented reality content included in the remote collaboration data by using the mapping information to the real world image included in the remote collaboration data, by the arbitrary terminal can

The augmented reality service providing system and method according to the present invention enables more efficient remote collaboration by allowing remote users to share augmented reality images in real time.

In addition, the augmented reality service providing system and method according to the present invention stores remote collaboration contents as data, but by separating or combining the real image and the augmented reality content and storing the data, it is possible to increase the possibility of data utilization.

In addition, the augmented reality service providing system and method according to the present invention provides the remote collaboration contents stored as data to a participant or a third party of the remote collaboration, so that the remote collaboration contents can be easily and efficiently shared.

1 is a diagram showing the structure of an augmented reality service providing system according to an embodiment.
2 is a flowchart illustrating a method of providing an augmented reality service according to an embodiment.
3 is a diagram illustrating a data structure of remote collaboration data according to an embodiment.
4 is a diagram illustrating a field configuration of remote collaboration data according to an embodiment.
5 is a diagram illustrating a field configuration of remote collaboration data according to another embodiment.
6 is a diagram for explaining a method of synthesizing augmented reality content into an image by using a marker in the image.
7 is a block diagram illustrating the structure of a server according to an embodiment.
8 is a block diagram illustrating the structure of a terminal according to an embodiment.

In describing the embodiments of the present specification, if it is determined that a detailed description of a related well-known configuration or function may obscure the gist of the present specification, the detailed description may be omitted.

As used herein, "comprises," "may include." The expression such as indicates the existence of the disclosed corresponding function, operation, component, etc., and does not limit one or more additional functions, operations, components, and the like. Also, in this specification, "includes." Or "have." The term such as is intended to designate that there is a feature, number, step, action, component, part, or combination thereof described in the specification, but one or more other features or number, step, action, component, part or It should be understood that it does not preclude the possibility of the existence or addition of combinations thereof.

In this specification, the singular expression includes the plural expression unless the context clearly dictates otherwise.

Hereinafter, the present invention will be described with reference to the accompanying drawings.

1 is a diagram showing the structure of an augmented reality service providing system according to an embodiment.

Referring to FIG. 1 , a system 1 according to the present invention includes a communication server 10 providing a remote collaboration service using a virtual environment, a plurality of terminals 20 and 30 receiving the remote collaboration service, and remote collaboration contents. It may include a storage server 40 for storing the data.

The communication server 10 is provided to provide a remote collaboration service using a virtual environment according to the present invention. The communication server 10 may be a network server, an application server, a domain server, etc. operated by a provider of an augmented reality service.

The communication server 10 may mediate remote collaboration between the plurality of terminals 20 and 30 . For example, when a request for providing an augmented reality service is received from the terminals 20 and 30 , the communication server 10 may proceed with remote collaboration between the terminals 20 and 30 . The communication server 10 transmits the real image captured by the first terminal 20 to the second terminal 30 in real time while remote collaboration is performed, and the augmented reality content generated by the second terminal 30 in real time can be transmitted to the first terminal 20 . The augmented reality content delivered to the first terminal 20 may be synthesized with a real image captured by the first terminal 20 and displayed as an augmented reality image.

The terminals 20 and 30 may receive an augmented reality service by performing data communication with the communication server 10 . The terminals 20 and 30 may be electronic devices such as mobile phones, smart phones, personal digital assistants (PDAs), tablets, computers, laptops, or wearable devices such as head mounted displays (HMDs), watches, and smart glasses. can

The terminals 20 and 30 may share data for generating an augmented reality image through the communication server 10 . For example, when the first terminal 20 shoots a real image and transmits the captured image to the second terminal 30 through the communication server 10, the second terminal 30 responds to the received image. may generate augmented reality content to be transmitted to the first terminal 20 . The first terminal 20 may display the augmented reality content received from the second terminal 30 by synthesizing it with the captured image in real time.

In an embodiment, the content of remote collaboration between the first terminal 20 and the second terminal 30 may relate to usage of various devices such as medical equipment, home appliances, and exercise equipment. For example, the first terminal 20 and the second terminal 30 may include an electrical impedance tomography apparatus, an X-ray apparatus, an ultrasound examination apparatus, a CT examination apparatus, an MRI examination apparatus, an endoscope apparatus, and an electrocardiogram apparatus. , blood pressure monitor, oximeter, body temperature monitor, respiratory system, blood analyzer, urine analyzer, body fluid composition analyzer, etc.

In this embodiment, any one of the terminals 20 and 30 (eg, the first terminal 20) is a terminal of a user (or purchaser) of the corresponding medical equipment, and the other (eg, the first terminal 20) The second terminal 30 ) may be a terminal of a manager (or a manufacturer or a seller) of the corresponding medical equipment. When a user photographs medical equipment by using a terminal and transmits an image of the photographed medical equipment to the terminal of the manager, the manager may input information on how to use the medical equipment to the terminal in response to the received image. The information input by the administrator may be generated as augmented reality content in the terminal and transmitted to the user's terminal. The user's terminal may synthesize and display the received augmented reality content on the image of the medical equipment in real time. The user can then use the medical equipment correctly under the supervision and direction of the administrator.

However, the present embodiment is not limited to the above, and the following embodiments may be applied to various augmented reality services.

The storage server 40 is provided to store and manage the contents of remote collaboration performed between the terminals 20 and 30 as data. The storage server 40 may receive the image of the real world captured by the first terminal 20 and the augmented reality content generated by the second terminal 30 from the communication server 10 , and store them as data. According to implementation, the storage server 40 may directly communicate with the terminals 20 and 30 to directly receive images of the real world and augmented reality content from the terminals 20 and 30 . In addition, the storage server 40 may share data stored with respect to various terminals including the terminals 20 and 30 that have performed the corresponding remote collaboration.

The storage server 40 receives images and augmented reality content of the real world from the communication server 10 in real time from the communication server 10 while remote collaboration of the terminals 20 and 30 is performed, or after the remote collaboration ends, the communication server It is possible to receive an image and augmented reality content of the real world from (10).

Meanwhile, although the communication server 10 and the storage server 40 are illustrated as separate components in FIG. 1 , the present embodiment is not limited thereto. That is, in other embodiments, the communication server 10 and the storage server 40 may be configured as one mediation server, and in this embodiment, the mediation server mediates remote collaboration between the terminals 20 and 30 and In addition, the contents of remote collaboration can be saved as data.

2 is a flowchart illustrating a method of providing an augmented reality service according to an embodiment. The flowchart of FIG. 2 shows an operation after the communication server 10 establishes a communication service for mediating communication between the terminals 20 and 30 at the request of the terminals 20 and 30 . The terminals 20 and 30 perform remote collaboration using an augmented reality service provided through the communication server 10 .

First, the first terminal 20 may capture an image of the real world and transmit the captured image to the communication server 10 in real time ( 101 ). In an embodiment, the image of the real world may include medical equipment.

The communication server 10 transmits the image of the real world received from the first terminal 20 to the second terminal 30 in real time (102).

The received image may be displayed in the second terminal 30 . In an embodiment, the second terminal 30 may receive a user input for generating augmented reality content in response to a displayed image. The user input may be an input using an input interface such as a mouse or a keyboard, or a touch input using a human body or a stylus pen. The user input may include text, a picture, an emoticon, a file, and the like. The user input may be input to an image in real time while the image is being reproduced in the second terminal 30 , or may be input to an arbitrary position in the image.

A user input for generating augmented reality content may relate to usage of medical equipment included in an image of the real world. For example, the user input may be information about a moving direction of a component, a manipulation method of an input interface, manipulation timing, and the like.

When the user input is received, the second terminal 30 may generate augmented reality content for the image ( 103 ). For example, the second terminal 30 may generate a text, a picture, an emoticon, etc. received through a user input as a graphic object. When the user inputs an arbitrary file (eg, file data having an extension such as pdf, jpg, gif and/or file data written in Word, Excel, PowerPoint, etc.) through user input, the second terminal 30 ) can create a file object received through user input. The graphic object and the file object may be stored separately in separate storages.

The second terminal 30 may transmit the generated augmented reality content to the communication server 10 ( 104 ). The communication server 10 transmits the augmented reality content received from the second terminal 30 to the first terminal 20 in real time ( 105 ).

The first terminal 20 may generate an augmented reality image by synthesizing the augmented reality content received from the communication server 10 with an image of the real world captured by the first terminal 20 in real time, and display it. Through this method, the user of the first terminal 20 and the user of the second terminal 30 may perform collaboration by sharing the augmented reality image remotely.

In various embodiments, the communication server 10 transmits the image of the real world received from the first terminal 20 and the graphic image received from the second terminal 30 to the storage server 40 while remote collaboration is performed. do (106). In an embodiment, the communication server 10 may deliver images and augmented reality content to the storage server 40 in real time while remote collaboration is being performed. In another embodiment, the communication server 10 temporarily stores the image and augmented reality content while remote collaboration is performed, and then temporarily stores the image and augmented reality content to the storage server 40 after the remote collaboration is terminated. can transmit

The storage server 40 may generate, store, and manage remote collaboration data using the image and augmented reality content received from the communication server 10 . In an embodiment, when there is a request from the terminals 20 and 30 or other third terminals participating in remote collaboration, the storage server 40 may provide the stored remote collaboration data. The terminal receiving the remote collaboration data can check the remote collaboration contents by playing the received remote collaboration data.

Hereinafter, a method of storing remote collaboration data in the storage server 40 will be described in more detail.

3 is a diagram illustrating a data structure of remote collaboration data according to an embodiment.

In an embodiment, the storage server 40 may generate one image by synthesizing the augmented reality content with the image, and store the generated image as remote collaboration data. In another embodiment, the storage server 40 may store the image and the augmented reality content separately as shown in FIG. 3 .

Referring to FIG. 3 , the storage server 40 may separately store an image and augmented reality content corresponding to the image. Also, according to an embodiment, the augmented reality content may be classified into a file object and a graphic object and stored.

In this embodiment, the storage server 40 may assign an identifier for identifying each image and the augmented reality content to each image and the augmented reality content. The storage server 40 may map an arbitrary image and augmented reality content corresponding thereto using the assigned identifier.

Also, the storage server 40 may store time information indicating a time at which augmented reality content should be synthesized into an image and location information indicating a location together. Time information and location information are mapping information of augmented reality content, and as shown in FIG. 3 , may be classified and stored separately from images and augmented reality content. The stored mapping information may be used to synthesize the augmented reality content at the correct position in the correct playback time period (frame) when synthesizing the corresponding augmented reality content into an image. A method of synthesizing augmented reality content into an image using mapping information will be described in more detail below with reference to FIG. 6 .

In an embodiment, the time information may be set as a value of a playback time section in an image. Alternatively, the time information may be set as a frame value.

In an embodiment, the location information may include at least one of a coordinate value for an arbitrary reference point in an image or information about an anchor. When generating location information using an anchor, at least one object identified at a location where the corresponding augmented reality content is to be synthesized may be used as the anchor.

The anchor may be, for example, at least a portion of a medical device identified through image analysis, a QR code, an image, text, a pattern, or a part of a human body (eg, a finger, etc.). An object used as an anchor is a Canny edge detector algorithm, a Harris corner detection algorithm, a Haar-like feature algorithm, a HOG (Histogram of Oriented Gradient) algorithm, or an algorithm based on deep learning. It can be detected through various object detection algorithms, such as

Identification of such an anchor may be performed when generating remote collaboration data in the storage server 40 , but is not limited thereto. For example, when augmented reality content is generated in the second terminal 30 and/or while the communication server 10 mediates remote collaboration, an anchor in the second terminal 30 and/or the communication server 10 is can be identified. In this embodiment, the second terminal 30 and the communication server 10 may transmit information about the identified anchor together when transmitting the image and/or digital data to another device.

4 is a diagram illustrating a field configuration of remote collaboration data according to an embodiment, and FIG. 5 is a diagram illustrating a field configuration of remote collaboration data according to another embodiment.

4 and 5 show the field configuration of the remote collaboration data generated according to the above. 4 and 5 , the remote collaboration data includes, for each image, identification information (ID) of a corresponding image, identification information (GID) of augmented reality content corresponding to the image, and object type of the corresponding augmented reality content. , the augmented reality content may include time information and location information to be synthesized. In the embodiment of FIG. 4 , the position information is composed of coordinate values for an arbitrary reference point in the image, and in the embodiment of FIG. 5 , the position information is composed of information of a marker on a point to which a corresponding graphic object is to be combined.

The stored remote collaboration data may be provided to the parties or a third party as needed and used appropriately. For example, remote collaboration data may be used not only for data storage, but also as evidence for legal problems that may arise in the future in the use of medical equipment.

6 is a diagram for explaining a method of synthesizing augmented reality content into an image by using a marker in the image.

In an embodiment, when the storage server 40 generates remote collaboration data as one image by synthesizing the augmented reality content with the image, the storage server 40 may transmit the requested image to the terminal. The terminal receiving the remote collaboration data may simply play the received image.

On the other hand, when the storage server 40 separates and stores the image and the augmented reality content, the storage server 40 identifies the corresponding augmented reality content and mapping information using the identifier of the requested image, and By determining the type, the stored augmented reality content and mapping information may be loaded. The storage server 40 may integrate the loaded data and provide it to the terminal as remote collaboration data.

The terminal receiving the remote collaboration data may synthesize augmented reality content into an image by using mapping information included in the remote collaboration data. Specifically, the terminal, as shown in FIG. 6 , the anchor A1 indicated in the image IM of the real world at the indicated time (frame) for any augmented reality content G1, G2, G3, A2, A3) can be identified. The terminal may synthesize augmented reality contents G1, G2, and G3 corresponding to the identified anchor positions A1, A2, and A3.

While the video is being played, the terminal can track the anchor through real-time video analysis, and display the augmented reality content by synthesizing the anchor at the location of the anchor for a designated time.

7 is a block diagram illustrating the structure of a server according to an embodiment. The server 50 shown in FIG. 7 may be the communication server 10 or the storage server 40 described above. Alternatively, the server 50 shown in FIG. 7 may be a single intermediary server that integrates the functions of the communication server 10 and the storage server 40 .

Referring to FIG. 7 , the server 50 may include a communication unit 51 , a control unit 52 , and a storage unit 53 .

The communication unit 51 may communicate with other external devices through a network, for example, the terminals 20 and 30 described with reference to FIG. 1 or other servers. For example, the communication unit 51 may communicate with the outside using a wired communication method or a wireless communication method such as Wibro, HSDPA, GSM, CDMA, Wi-Fi, 3G 4G LTE, 5G, or the like.

The communication unit 51 may convert the transmitted data through the control unit 52 into an analog signal of a transmittable form and transmit it to the outside, process the data received from the outside in a digital form, and transmit it to the control unit 52 .

The controller 52 may control the overall operation of the server 50 . In an embodiment, the control unit 52 may process the image and augmented reality content received through the communication unit 51 and transmit it to an external device or store it in the storage unit 53 .

In an embodiment, the controller 52 may generate remote collaboration data by processing an image and augmented reality content received through the communication unit 51 . For example, the controller 52 may generate one image by synthesizing the augmented reality content with the received image, and store the generated image in the storage unit 53 as remote collaboration data. In another embodiment, the controller 52 may allocate storage for the image and the augmented reality content to the storage unit 53 , respectively, and store the image and the augmented reality content separately.

In this embodiment, the controller 52 may assign an identifier (ID) for identifying each image and the augmented reality content to each image and the augmented reality content. In addition, the storage control unit 52 generates time information indicating the time at which the augmented reality content should be synthesized in the image and location information indicating the location, and maps the image and the augmented reality content identifier to the storage unit 53 . can be saved

In an embodiment, the location information may include at least one of a coordinate value for an arbitrary reference point in an image or information about an anchor. When generating location information using an anchor, the controller 52 may identify an arbitrary object at a location where augmented reality content is to be synthesized through image analysis, and set the identified object as an anchor.

The storage unit 53 may store data under the control of the control unit 52 . The storage unit 53 may store an operating system, a program, software, etc. necessary for driving the server 50 . Also, the storage unit 53 may store the remote collaboration data generated by the control unit 52 . To this end, the storage unit 53 includes a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, SD or XD memory). etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), It may include a storage medium of at least one type of a magnetic memory, a magnetic disk, and an optical disk.

8 is a block diagram illustrating the structure of a terminal according to an embodiment. The terminal 60 illustrated in FIG. 8 may be the terminals 20 and 30 described above.

Referring to FIG. 8 , the terminal 60 may include a camera unit 61 , an input unit 62 , a display unit 63 , a communication unit 64 , a control unit 65 , and a storage unit 66 .

The camera unit 61 processes image frames such as still images or moving images obtained by the image sensor in the shooting mode. In various embodiments of the present disclosure, the camera unit 61 may include a depth camera capable of identifying the shape and depth of the user's real space (subject). In addition, the camera unit 61 may be configured to include an infrared camera for photographing a user's hand gesture and the like. The processed image frame may be displayed on the display unit 63 or stored in the storage unit 66 .

In an embodiment, the camera unit 61 captures an image of the real world according to a user's request. At least one object such as medical equipment, home appliances, and exercise equipment may be included in the captured image.

The input unit 62 receives various inputs from the user. The input unit 62 may include a mechanical input unit such as a mouse, a keyboard, a touch pad, a key pad, a jog dial, a dome switch, a button, or a virtual input unit displayed on the display unit 63 . The virtual input means may consist of, for example, graphics, text, icons, video, or a combination thereof.

In an embodiment, the input unit 62 may receive a user input to generate augmented reality content for the image being reproduced through the display unit 63 . The user input may be input to the image in real time while the image is being reproduced on the display unit 63 , or may be input to an arbitrary position in the image.

The display unit 63 may visually display various pieces of information processed by the terminal 60 . The display unit 63 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED) display, and a flexible display. display) and at least one of a three-dimensional display (3D display).

The communication unit 64 may communicate with another external device, for example, the communication server 10 described with reference to FIG. 1 through a network. For example, the communication unit 64 may communicate with the outside using a wired communication method or a wireless communication method such as Wibro, HSDPA, GSM, CDMA, Wi-Fi, 3G 4G LTE, 5G, or the like.

The communication unit 64 converts the data transmitted through the control unit 65 into an analog signal in a transmittable form and transmits it to the outside, and processes the data received from the outside in a digital form and transmits the data to the control unit 65 .

The controller 65 is configured to control each component of the terminal 60 in order to receive the augmented reality service according to the present invention and perform remote collaboration with an external terminal.

In an embodiment, the controller 65 may generate an augmented reality image by synthesizing an image of the real world generated by the terminal 60 or received from the outside through the communication unit 64 and the augmented reality content. The controller 65 may generate the augmented reality image by synthesizing the augmented reality content at the indicated position at the indicated time of the image by using the time information and the location information on the augmented reality content. In an embodiment, the location information may be information about a coordinate value or an anchor for an arbitrary reference point in an image. When the location information consists of information on the anchor, the controller 65 may identify the anchor in the image and synthesize augmented reality content at the identified anchor location.

The storage unit 66 may store data under the control of the control unit 65 . The storage unit 66 may store an operating system, a program, software, etc. necessary for driving the terminal 60 . Storage unit 66 is a flash memory type (flash memory type), hard disk type (hard disk type), multimedia card micro type (multimedia card micro type), card type memory (for example, SD or XD memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, It may include a storage medium of at least one type of a magnetic disk and an optical disk.

Those of ordinary skill in the art to which the present invention pertains will be able to make various modifications and variations without departing from the essential characteristics of the present invention. In addition, the embodiments disclosed in the present specification and drawings are merely provided for specific examples to easily explain the contents of the present invention and help understanding, and are not intended to limit the scope of the present invention. Therefore, the scope of the present invention should be construed as including all changes or modifications derived based on the technical spirit of the present invention in addition to the embodiments disclosed herein are included in the scope of the present invention.

1: Sales system
10: communication server
20, 30: terminal
40: storage server

Claims (19)

As an augmented reality service providing system,
a communication server that mediates remote collaboration between terminals located in remote locations;
a first terminal for capturing an image of the real world and transmitting it to the communication server in real time;
a second terminal that generates augmented reality content for the image of the real world received through the communication server and transmits it in real time to the first terminal through the communication server;
On the basis of the image of the real world and the augmented reality content received through the communication server, comprising a storage server for storing remote collaboration data including the content of the remote collaboration,
The first terminal,
A system for synthesizing the augmented reality content with the image of the real world and displaying it as an augmented reality image. According to claim 1, wherein the storage server,
A system for generating a single image by synthesizing the augmented reality content with the image of the real world, and storing the generated image as the remote collaboration data. According to claim 1, wherein the storage server,
A system that separates and stores the image of the real world and the augmented reality content in separate storages. According to claim 2, wherein the augmented reality content,
contains graphic objects and file objects;
The storage server,
Separately storing the graphic object and the file object in separate storage, the system. According to claim 3, The storage server,
The system further stores as the remote collaboration data by generating mapping information including time information and location information about the image of the real world in which the augmented reality content is to be synthesized. According to claim 5, wherein the time information,
The system is set as a playback time interval value or a frame value for the real-world image. According to claim 5, wherein the location information,
The system is set as a coordinate value for an arbitrary reference point in the image of the real world or information about an anchor identified in the image of the real world. The method of claim 7, wherein the anchor,
any object identified through image analysis within the real-world image. According to claim 1, wherein the storage server,
A system for transmitting the remote collaboration data to any terminal including the first terminal and the second terminal. 10. The method of claim 9, wherein the arbitrary terminal,
A system for sharing the contents of the remote collaboration by displaying the augmented reality image by synthesizing the augmented reality content included in the remote collaboration data using mapping information to the real world image included in the remote collaboration data. According to claim 1, wherein the image of the real world,
At least one medical device comprising an image,
The augmented reality content is
and information for guiding the use of the at least one medical device. As a method of providing an augmented reality service,
The first terminal capturing the image of the real world and transmitting it to a communication server;
transmitting, by the communication server, the image of the real world to a second terminal located at a remote location from the first terminal;
generating, by the second terminal, augmented reality content for the image of the real world and transmitting it to the communication server;
transmitting, by the communication server, the augmented reality content to the first terminal;
displaying, by the first terminal, as an augmented reality image by synthesizing the augmented reality content with the image of the real world;
transmitting, by the communication server, the image of the real world and the augmented reality content to the storage server to the storage server; and
and storing, by the storage server, remote collaboration data based on the image of the real world and the augmented reality content. The method of claim 12, wherein the storing of the remote collaboration data comprises:
generating one image by synthesizing the augmented reality content with the image of the real world; and
and storing the generated image as the remote collaboration data. The method of claim 12, wherein the storing of the remote collaboration data comprises:
generating a graphic object or a file object according to the type of the augmented reality content; and
and separately storing the image of the real world, the graphic object, and the file object in separate storages. The method of claim 14, wherein the storing of the remote collaboration data comprises:
generating mapping information including time information and location information on the image of the real world in which the augmented reality content is to be synthesized; and
The method of storing the mapping information in a separate storage corresponding to the image of the real world. The method of claim 15, wherein the time information,
The method is set as a playback time interval value or a frame value for the image in the real world. The method of claim 15, wherein the generating of the mapping information comprises:
identifying an anchor in the image of the real world with respect to a position where the augmented reality content is to be synthesized in the image of the real world; and
and setting information about the anchor as the location information. 13. The method of claim 12,
The method further comprising the step of the storage server transmitting the remote collaboration data to any terminal including the first terminal and the second terminal. 19. The method of claim 18,
The method further comprising the step of displaying the augmented reality image by synthesizing the augmented reality content included in the remote collaboration data by using the mapping information to the real world image included in the remote collaboration data, by the arbitrary terminal.

Images