KR101525936B1 - Method and Apparatus for Providing Augmented Reality Service Based on SNS - Google Patents

Abstract

Disclosed are an apparatus and method for providing an augmented reality service in a social network environment. More specifically, the apparatus and method for providing an augmented reality service in a social network environment allow a user to log in a specific application and add location information of friends who are logged in the social network to photographed actual images based on a GPS signal so as to provide an augmented reality image.

Description

[0001] The present invention relates to an apparatus and a method for providing augmented reality service in a social network environment,

The present invention relates to an apparatus and method for providing an augmented reality service for outputting an augmented reality image by adding friend location information logged in a social network in a social network environment.

The contents described in this section merely provide background information on the present embodiment and do not constitute the prior art.

Augmented reality is a technique of superimposing virtual objects on the real world seen by the user. An augmented reality application is being developed which provides augmented reality service based on information provided on the Internet. These augmented reality applications require much development, such as location-based AR applications, AR applications based on image recognition, and next generation AR applications.

In addition, through the development and dissemination of smartphone terminals, it is inevitable that the development of virtual social and augmented reality, smartphone social network, and smartphone social network with augmented reality is absolutely necessary. Conventional marker-based augmented reality systems require a lot of processing frequency and storage space required for driving, and it is necessary to know how the GPS operates in the terminal so that the acquisition and storage of location-based information can be smoothly proceeded, The use of Augmented Reality has a problem of high battery consumption.

The present embodiment provides augmented reality service in a social network environment in which an augmented reality image is provided by adding friend location information logged in a social network based on a GPS signal to an actual image captured after performing login for a specific application The present invention has been made to solve the above-mentioned problems occurring in the prior art.

According to an aspect of the present invention, a log-in processing unit for acquiring log-in information for a specific application and activating a camera provided when the log-in information is authenticated;

An image acquiring unit acquiring actual image information from the camera driven; A position processor for receiving GPS signals to generate current position information, and generating augmented reality position related information based on current position information and predetermined augmented reality position information; An augmented reality control unit for acquiring friend location information on a logged-in friend in cooperation with a social network server, and controlling the virtual extracted information generated based on the friend location information and the augmented reality location related information to be fused to the actual image information; And an output unit outputting the augmented reality image through a display unit including the virtual extraction information and the actual image information.

According to another aspect of the present invention, a login process for acquiring login information for a specific application and driving a camera provided when the login information is authenticated; An image acquiring step of acquiring actual image information from the camera driven; A position processing step of receiving GPS signals to generate current position information, and generating augmented reality position related information based on current position information and predetermined augmented reality position information; An augmented reality control process for acquiring friend location information for a logged-in friend in cooperation with a social network server, and controlling the virtual extracted information generated based on the friend location information and the augmented reality location related information to be fused with the actual image information ; And outputting the augmented reality image through the display unit including the augmented reality image fused with the virtual extraction information and the actual image information, to provide augmented reality service in a social network environment .

As described above, according to the present embodiment, by providing the augmented reality image by adding the friend location information logged in the social network based on the GPS signal, the processing frequency required for the operation and the storage space There is an effect of solving the disadvantages of a large consumption.

In addition, since the augmented reality service is provided in a social network environment using the GPS signal, the accuracy of the augmented reality service can be improved.

FIG. 1 is a block diagram illustrating an augmented reality system that operates in conjunction with a social network server according to an embodiment of the present invention. Referring to FIG.
2 is a block diagram schematically illustrating a terminal for providing an augmented reality service in a social network environment according to an embodiment of the present invention.
3 is a flowchart illustrating a method of providing an augmented reality service in a social network environment according to an embodiment of the present invention.
4 is a flowchart for explaining the operation of the augmented reality development tool (ARKit) to provide the augmented reality service in the social network environment according to the present embodiment.
5 is a diagram illustrating a structure of a class for providing an augmented reality service in a social network environment according to an embodiment of the present invention.
6 is a diagram illustrating an operation of displaying friend location information using an augmented reality service in a social network environment according to the present embodiment.
7 is a diagram illustrating an operation of transmitting a message using an augmented reality service in a social network environment according to an embodiment of the present invention.

Hereinafter, the present embodiment will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an augmented reality system that operates in conjunction with a social network server according to an embodiment of the present invention. Referring to FIG.

The augmented reality system in cooperation with the social network server according to the present embodiment includes a terminal 110 and a social network server 120. [ The augmented reality system shown in Fig. 1 is according to one embodiment, and not all of the blocks shown in Fig. 1 are essential elements, and in some embodiments, some blocks may be added, changed or deleted.

The terminal 110 is an apparatus that can transmit and receive various data according to a key operation or a command of a user and provides an augmented reality service. The terminal 110 may be a tablet PC, a laptop, a personal computer Computer, a smart phone, a personal digital assistant (PDA), and a mobile communication terminal.

The terminal 110 may operate a pre-installed social network application, and the social network application may be connected to the social network server 120 to perform a service for outputting various information registered by a user and a friend of the user, And can be provided in connection with the augmented reality service.

Hereinafter, a process of providing the augmented reality service in the social network environment of the terminal 110 according to the present embodiment will be described.

The terminal 110 acquires log-in information for a specific application through a user's input, and operates the camera when the log-in information is authenticated. Here, the specific application may be an application for a social network such as Twitter or Facebook, or may be an application linked to a server providing various information such as map information, traffic information, and knowledge information. Here, the terminal 110 may perform an operation of initializing the actual image information acquired from the driven camera.

The terminal 110 receives the current position information from the GPS satellite, and based on the received current position information, acquires an augmented reality position such as a moving angle, an azimuth angle, a radius, etc. obtained using an augmented reality kit (ARKit) Information on the augmented reality location such as the terminal location information, the direction information, and the search range information.

The terminal 110 acquires friend location information for a friend set to share location information from the social network server 120, compares the augmented reality location related information with the friend location information, and stores the friend location information in the search range information And outputs the augmented reality image by merging the friend location information with the actual image information. Here, the friend location information output by being fused to the actual image information may include information such as a street name, a place name, a friend name, and a friend ID of a place previously stored by a friend.

Also, the terminal 110 may send a message to the friend based on the friend location information output in the output augmented reality image, or may input additional information into the friend location information.

The social network server 120 refers to a device that interworks with a plurality of terminals via a network and transmits / receives / stores / manages various data to / from a social network.

The social network server 120 transmits the friend location information of the logged-in friend to the terminal 110 based on the user's request. Here, the social network server 120 may provide friend location information according to a user's request. However, based on the user's request, the social network server 120 extracts, based on the existence of previously registered posts, Friend location information may be provided to the terminal 110.

Meanwhile, in the present embodiment, the social network server 120 transmits / receives and stores only information related to the social network. However, the present invention is not limited to this, and various information such as map information, traffic information and knowledge information may be transmitted / Server.

2 is a block diagram schematically illustrating a terminal for providing an augmented reality service in a social network environment according to an embodiment of the present invention.

The terminal 110 according to the present embodiment includes a login processing unit 210, an image obtaining unit 220, a position processing unit 230, an augmented reality control unit 240, and an output unit 250. The terminal shown in FIG. 2 is according to one embodiment, and not all of the blocks shown in FIG. 2 are essential elements, and in some embodiments, some blocks may be added, changed or deleted.

The components of the terminal 110 according to the present embodiment are connected to each other via a bus. Here, the bus is a communication path for connecting an ECU (Electronic Control Unit), a node, a module, and the like in the device. The bus includes an address bus, a data bus, a control bus, a field bus Bus, and the like, and may be implemented as a wired network of a star type, a mesh type, a ring type, a mesh type, a tree type, or the like, or may be implemented as a wireless network structure. These buses can be implemented with bus systems such as CAN (Controller Area Network), RS485 bus, Verse Module Eurocard (VME) bus, Multibus, and the next generation bus system to be implemented and various network structures of wired and wireless.

The login processing unit 210 acquires login information for a specific application through a user input, and operates the camera provided in the terminal 110 when the login information is authenticated. Here, the specific application may be an application for a social network such as Twitter or Facebook, or may be an application linked to a server providing various information such as map information, traffic information, and knowledge information.

The image acquiring unit 220 acquires actual image information photographed by the camera. Here, the image acquiring unit 220 acquires actual image information using video streaming of a camera, and the image acquiring unit 220 may include a high-end CCD camera or the like, and may acquire actual image information accurately in real time It can be implemented in various forms if it supports image precision and acquisition speed.

The position processor 230 receives the current position information from the GPS satellite and generates the augmented reality position related information such as the terminal position information, the direction information, and the search range information using the received current position information and predetermined augmented reality position information . Here, the position processor 230 includes a GPS information processor 232 and an augmented reality position information processor 234.

The GPS information processing unit 232 generates current position information of the terminal 110 in cooperation with the GPS satellite and transmits the generated current position information to the augmented reality position information processing unit 234. [ Here, the GPS information processing unit 232 may generate current position information only at the received GPS coordinates using GPS satellites, but the present invention is not limited thereto. The acceleration information received from the provided acceleration sensor may be combined with the GPS coordinates The current position information may be calculated.

The augmented reality location information processing unit 234 performs an operation of generating augmented reality location related information using an augmented reality kit (ARKit). To be more specific, the augmented reality position information processing unit 234 applies the augmented reality position information, such as the movement angle, the azimuth angle, and the radius, obtained using the augmented reality development tool based on the current position information, , Search range information, and the like. Here, the augmented reality development tool is preferably implemented in software, but is not limited thereto, and may be implemented with hardware (e.g., augmented reality development module) for generating position related information on the augmented reality.

The augmented reality position information processing unit 234 may generate the augmented reality position related information by calculating the azimuth based on the current position information. For example, the augmented reality location information processing unit 234 is the latitude, φ ₁ ', the hardness is zero latitude in the present position information "φ _2", a hardness of "L (Positive Eastward), the azimuth of the location expression (1) And the augmented reality position related information can be generated based on the calculated azimuth angle.

(A: azimuth, φ ₁ : latitude of current location information, φ ₂ : latitude moved based on current location information)

The augmented reality control unit 240 acquires the friend location information about the friend who is logged in from the social network server 120 and stores the virtual extraction information generated based on the friend location information and the augmented reality location related information And performs the control operation. To be more specific, the augmented reality controller 240 acquires friend position information on a predetermined friend from the social network server 120 so as to share position information, and acquires search range information and friend position information among the augmented reality position related information If the search range information includes the friend location information, the friend location information is fused to the actual image information to control the augmented reality image to be output. Here, the friend location information displayed in the augmented reality image may include information such as a street name, a place name, a friend name, and a friend ID of a place previously stored by a friend. For example, the augmented reality controller 240 obtains the location information of 'A' from the social network server 120, and if the location information of 'A' corresponds to the search range information, the name of 'A' , The coordinate value is overlaid in the search range information by using the place name registered in the social network server 120 in advance. In addition, the augmented reality controller 240 updates the augmented reality location related information after storing the attributes such as the distance and inclination of the previously registered place, confirms the angle (range) and the azimuth angle that can be fused to the actual image information, 'Or the name of the previously registered place to the actual image information so that the augmented reality image can be outputted.

Also, the augmented reality controller 240 may transmit a message to the friend based on the friend position information output in the output augmented reality image, or may input additional information into the friend position information. Here, the message may be message information transmitted via the social network server 120, but the present invention is not limited thereto. For example, the friend information (e.g., friend information) stored in the post or friend location information posted on the social network server 120 Telephone number, etc.) to the communication terminal.

The output unit 250 outputs the augmented reality image obtained by fusing the friend's location information (i.e., virtual extraction information) to the actual image information using the display unit.

3 is a flowchart illustrating a method of providing an augmented reality service in a social network environment according to an embodiment of the present invention.

The terminal 110 acquires and authenticates login information for a specific application through the input of the user (S310), and activates the provided camera (S320). Here, the specific application may be an application for a social network such as Twitter or Facebook, or may be an application linked to a server providing various information such as map information, traffic information, and knowledge information.

The terminal 110 acquires the actual image information photographed from the provided camera.

The terminal 110 receives the current position information from the GPS satellite and applies the augmented reality position information such as the moving angle, the azimuth angle, and the radius acquired using the augmented reality development tool based on the received current position information (S330) Information such as terminal position information, direction information, and search range information (S340).

The terminal 110 acquires the friend location information about the logged-in friend in cooperation with the social network server 120, compares the friend location information and the augmented reality location related information (S350) If the information includes the friend location information, the friend location information is fused to the actual image information to output the augmented reality image (S360). For example, the augmented reality controller 240 obtains the location information of 'A' from the social network server 120, and if the location information of 'A' corresponds to the search range information, the name of 'A' , The coordinate value is overlaid in the search range information by using the place name registered in the social network server 120 in advance. In addition, the augmented reality controller 240 updates the augmented reality location related information after storing the attributes such as the distance and inclination of the previously registered place, confirms the angle (range) and the azimuth angle that can be fused to the actual image information, 'Or the name of the previously registered place can be fused to the actual image information and output.

As described above, the operation of the terminal 110 according to the present embodiment described in FIG. 3 can be implemented by a program and recorded on a computer-readable recording medium. A program for implementing the operation of the terminal 110 according to the present embodiment is recorded, and a computer-readable recording medium includes all kinds of recording devices for storing data that can be read by a computer system. Examples of such computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc., and also implemented in the form of a carrier wave (e.g., transmission over the Internet) . The computer readable recording medium may also be distributed over a networked computer system so that computer readable code is stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present embodiment can be easily inferred by programmers in the technical field to which the present embodiment belongs.

4 is a flowchart for explaining the operation of the augmented reality development tool (ARKit) to provide the augmented reality service in the social network environment according to the present embodiment.

The process of generating the augmented reality position related information using the augmented reality development tool of the terminal 110 is as follows.

The terminal 110 drives the augmented reality development tool (S410), and generates current position information through GPS interworking (S420). Here, the terminal 110 may receive the GPS signal from the GPS satellite and generate the current position information. However, the terminal 110 may combine the acceleration information received from the acceleration sensor included in the terminal 110 with the GPS signal to calculate the current position information It is possible.

The terminal 110 obtains the augmented reality location information such as the moving angle, the azimuth angle, and the radius (S430) and applies the augmented reality location information such as the moving angle, the azimuth angle, and the radius acquired based on the current location information, , Direction information, search range information, and the like.

The terminal 110 renders the generated augmented reality location related information (S440), terminates the augmented reality development tool, and transmits the augmented reality location related information to the logged-in friend acquired from the social network server 120 And outputs the generated augmented reality image using the friend location information of the augmented reality service.

Here, the augmented reality development tool operating in the terminal 110 is preferably implemented in software, but is not limited thereto, and may be hardware (for example, hardware for generating position-related information for the augmented reality included in the terminal 110 : Augmented reality development module).

5 is a diagram illustrating a structure of a class for providing an augmented reality service in a social network environment according to an embodiment of the present invention.

The terminal 110 may implement a specific application in software in order to provide the augmented reality service in cooperation with the social network server 120.

As shown in FIG. 5, the terminal 110 may be implemented with a plurality of class structures for providing augmented reality services in a social network environment. For example, the class structure may be 'ApplicationController' which performs overall control of a specific application,

ARController 'and' RootViewController 'for controlling and outputting the ARF service,' CamController 'for controlling the camera provided in the terminal 110,' SocialContext 'and' Graph for friends' linked with the social network server 120 'CompassController', 'gpsController', and 'ARCoordinate' for processing information necessary for the location of the terminal 110 and the augmented reality.

6 is a diagram illustrating an operation of displaying friend location information using an augmented reality service in a social network environment according to the present embodiment.

As shown in FIGS. 6A and 6B, the terminal 110 acquires friend location information on a predetermined friend from the social network server 120 to share location information, The search range information is compared with the friend location information, and if the search range information includes the friend location information, the friend location information is fused to the actual image information and output. Here, the friend location information fused to the actual image information may include information such as a street name, a place name, a friend name, and a friend ID of a place stored in the social network server 120 by the friend.

For example, as shown in (a) of FIG. 6, the terminal 110 obtains location information on 'YooGul' logged in from the social network server 120, In the case where the location information is applicable, the coordinate value of the location where 'YooGul' is registered can be overlaid and the friend name (YooGul) can be fused to the actual image information and output.

6 (a), the terminal 110 obtains the location information of 'YooGul' logged in from the social network server 120 and stores location information of 'YooGul' in the search range information It is possible to overlay the coordinate value of the place where 'Yugel' is registered and to output a fusion ID ('denver') to the actual image information and output it.

7 is a diagram illustrating an operation of transmitting a message using an augmented reality service in a social network environment according to an embodiment of the present invention.

As shown in FIG. 7, the terminal 110 obtains the location information of 'YooGul' logged in from the social network server 120, and when the location information of 'YooGi' corresponds to the search range information, ('Denver') is overlaid on the coordinate value of the registered location of the user, and the message is transmitted through the friend ID in the output augmented reality image. Here, the message may be message information transmitted via the social network server 120, but the present invention is not limited thereto. For example, the friend information (e.g., friend information) stored in the post or friend location information posted on the social network server 120 Telephone number, etc.) to the communication terminal.

The foregoing description is merely illustrative of the technical idea of the present embodiment, and various modifications and changes may be made to those skilled in the art without departing from the essential characteristics of the embodiments. Therefore, the present embodiments are to be construed as illustrative rather than restrictive, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as being included in the scope of the present invention.

110: terminal 120: social network server
210: login processing unit 220:
230: Position processor 240: Augmented reality controller
250: Output section

Claims (13)

A login processing unit for acquiring login information for a specific application and driving the camera provided when the login information is authenticated;
An image acquiring unit acquiring actual image information from the camera driven;
A GPS information processing unit for receiving the GPS signal from a GPS satellite and generating current position information; And
Based on the current position information, the augmented reality position related information using at least one of a moving angle, an azimuth angle, and a radius acquired using an augmented reality kit (ARKit) A position processing unit including an augmented reality position information processing unit;
Acquiring from the social network server preset friend location information for sharing location information, comparing the search range information and the friend location information among the augmented reality location related information, and, when the friend location information is included in the search range information, An augmented reality control unit for generating and outputting an augmented reality image obtained by fusing virtual extraction information generated based on information to actual image information; And
And an output unit outputting the augmented reality image through the display unit including the virtual extraction information and the actual image information,
The position information processor may calculate the azimuth angle using the following equation

(Where A: azimuth angle,? 1: latitude of current location information, and? 2: latitude moved based on current location information)
Wherein the terminal is used in a social network environment.
delete The method according to claim 1,
The GPS information processing unit,
Acquiring acceleration information measured from an acceleration sensor and combining the acceleration information with the GPS signal to generate the current position information. The method according to claim 1,
Wherein the augmented reality position information processing unit comprises:
Wherein the augmented reality location information includes at least one of terminal location information, direction information, and search range information by calculating an azimuth based on the current location information, and generates the augmented reality location related information in a social network environment . The method according to claim 1,
Wherein the augmented reality position information processing unit comprises:
Related information including at least one of terminal location information, direction information, and search range information using the augmented reality location information obtained using the augmented reality development tool, A terminal that provides augmented reality service in the environment.
delete The method according to claim 1,
Wherein the friend location information comprises:
Wherein the at least one information includes at least one of a street name, a place name, a friend name, and a friend ID of a place pre-stored in the social network server. The method according to claim 1,
Wherein the augmented reality controller comprises:
And transmits the message based on the friend location information displayed in the augmented reality image. 9. The method of claim 8,
The message comprises:
Wherein the mobile terminal is one of a message transmitted via the social network server, a broadcast message stored in the social network server, and a mobile message transmitted to the communication terminal. The method according to claim 1,
The specific application,
Wherein the mobile terminal is a social network application that performs a social network service in cooperation with the social network server. A login process for acquiring login information for a specific application and driving a camera provided when the login information is authenticated;
An image acquiring step of acquiring actual image information from the camera driven;
A GPS information processing step of receiving the GPS signal from a GPS satellite and generating current position information; And
Based on the current position information, the augmented reality position related information using at least one of a moving angle, an azimuth angle, and a radius acquired using an augmented reality kit (ARKit) A position processing process including augmented reality position information processing process;
Acquiring from the social network server preset friend location information for sharing location information, comparing the search range information and the friend location information among the augmented reality location related information, and, when the friend location information is included in the search range information, An augmented reality control process for generating and outputting an augmented reality image fused with virtual image information generated based on information; And
And outputting the augmented reality image through a display unit including the augmented reality image fused with the virtual extraction information and the actual image information,
In order to measure the azimuth angle,

(Where A: azimuth angle,? 1: latitude of current location information, and? 2: latitude moved based on current location information)
The method comprising the steps of: (a) providing the augmented reality service in a social network environment;

delete delete

Images