KR20220011906A - Apparatus and method device for providing social network services using augmented reality - Google Patents

Abstract

The present invention relates to a social network service providing device using augmented reality. The social network service providing device using augmented reality comprises: an avatar management unit generating a user avatar corresponding to a user and storing it in a first storage unit; a screen control unit that overlaps and displays the user avatar on a real image displayed on a screen of a user terminal in response to a photographing target area when a photographing mode is detected; an image management unit for obtaining an augmented reality image in which the real image and the user avatar are combined and storing it in a second storage unit when a user input is made to a photographing button in the photographing mode; and an SNS management unit configured to upload the augmented reality image in an SNS relational network for sharing when a share request signal for sharing the augmented reality image with other users is input from the user.

Description

Device for providing social network service using augmented reality {APPARATUS AND METHOD DEVICE FOR PROVIDING SOCIAL NETWORK SERVICES USING AUGMENTED REALITY}

The present application relates to an apparatus for providing a social network service using augmented reality.

Social Networking Service (SNS, Social Media) refers to an online platform that creates and strengthens social relationships through free communication, information sharing, and network expansion between users. The most important part of SNS is that it creates, maintains, strengthens, and expands a social network through this service. It can be more meaningful when information is shared and distributed through such a network.

Recently, along with the increase of smartphone users and the expansion of wireless Internet services, the number of users of SNS is also rapidly increasing. The value of using SNS for marketing is emerging day by day in that it can identify and manage service users who can be classified into a broad and specific group at the same time by means of a database.

However, although the conventional SNS provides various functions, it constantly causes problems in various advertisements and personal privacy issues. In addition, the conventional SNS has a limitation in expressing information because it uses a daily sharing method limited to photo format and short video, which does not sufficiently satisfy their needs for some people with a desire to show off, and as time goes by, There is a problem that users gradually lose interest as the number increases.

The technology that is the background of the present application is disclosed in Korean Patent Publication No. 10-1652433.

The present application is intended to solve the problems of the prior art described above, and the problems of the conventional SNS (for example, inconvenient to use SNS due to various advertisements and infringing on personal privacy, daily sharing limited to photo formats and short videos) Problems that have limitations in expressing information using methods, problems that do not sufficiently satisfy their needs for some people with a desire to show off, problems that users lose interest in as time goes by without an element of interest, etc. ) for the purpose of providing an apparatus and method for providing a social network service using augmented reality that can solve the problem.

However, the technical problems to be achieved by the embodiments of the present application are not limited to the technical problems described above, and other technical problems may exist.

As a technical means for achieving the above technical problem, the apparatus for providing social network services (SNS, Social Network Services) using augmented reality according to an embodiment of the present application generates a user avatar corresponding to the user and includes a first storage unit Avatar management unit to store in; a screen control unit that overlaps and displays the user avatar on a real image displayed on the screen of the user terminal in response to a photographing target area when it is detected that the photographing mode is in the photographing mode; an image management unit for obtaining an augmented reality image in which the real image and the user avatar are combined and storing the augmented reality image in a second storage unit when a user input is made to the shooting button in the shooting mode; And when a share request signal for sharing the augmented reality image with other users is input from the user, it may include an SNS management unit for uploading the augmented reality image to an SNS relational network for sharing.

In addition, the user is a first user, the user avatar of the first user is a first user avatar, the augmented reality image is a first augmented reality image, and the screen controller includes: It is detected that the shooting mode is for shooting the same target region as the target region, and when the first user avatar exists as a user avatar pre-stored in relation to the shooting target region, the image is displayed on the screen of the user terminal of the second user. The second user avatar and the first user avatar may be overlapped and displayed on the displayed real image.

In addition, the image management unit, the real image, the second user avatar and the first user avatar is combined with the augmented reality image as an augmented reality image to obtain and store the second augmented reality image, but cannot be combined in relation to the first user avatar If the condition is preset, combining for obtaining the second augmented reality image may be performed in consideration of the uncombinable condition.

The avatar manager may generate the user avatar in consideration of body characteristic information received from the user, and the user avatar may include at least one of a user's body type, face shape, facial expression, and skin tone received as the body characteristic information. It may be a 3D avatar generated in consideration of

In addition, the avatar manager may be configured to obtain at least one of the uploaded information analysis result calculated by analyzing the user's biometric information, the weather forecast information, and the upload information uploaded by the user when it is detected that the shooting mode is in the shooting mode. and an information obtaining unit, configured to update status information of the user avatar based on the at least one obtained information, and to generate and store the updated user avatar.

In addition, when the real image is displayed on the screen of the user terminal for a preset time, the screen control unit displays at least one preselected recommended location on the real image, and among the at least one preselected recommended location The user avatar may be displayed at any one previously selected and recommended location.

In addition, the at least one preselected recommended location is a location selected in consideration of object information identified through image analysis of the real image displayed on the screen and/or a plurality of augmented reality images uploaded to the SNS relational network It may be a location selected in consideration of statistical location information related to the display location of the user avatar calculated based on the analysis of .

In addition, when the real image is displayed on the screen of the user terminal for a preset time, the screen controller analyzes the uploaded information uploaded to the SNS relational network in relation to the real image to a place located in the vicinity of the real image It is determined whether there is information related to a recommended place to visit, and if it is determined as a result of the determination, a manager avatar including information related to the recommended place to visit may be generated and displayed on the real image.

In addition, when a user input for the manager avatar is made, the screen control unit displays the recommended visit place related information in a pop-up form on one area of the screen of the user terminal, and includes a plurality of places within the recommended visit place related information. When the item is included, the display order of the plurality of place items in the recommended visit place related information displayed in the pop-up form may be updated in consideration of the level of user action information for the plurality of place items.

In addition, the level of the user action information is divided into a plurality of levels including a first level, a second level, and a third level, and the user action information corresponding to the first level is obtained from the plurality of place items from the user terminal. information on a user action of performing a selection input operation to select at least one place item from , information about a user action of performing a route guidance request input operation for requesting route guidance to a place corresponding to the one place item, and the user action information corresponding to the third level includes the route guidance request input It may be information about a user action of performing a place image acquisition operation of acquiring an image related to a place corresponding to the one place item based on the route guidance information generated in response to the operation.

On the other hand, the social network service providing system using augmented reality according to an embodiment of the present application, the social network service providing apparatus using the augmented reality according to the embodiment of the present application described above; and a user terminal interlocking with the apparatus for providing a social network service using the augmented reality to receive a control signal for controlling an operation of the apparatus for providing a social network service using the augmented reality as input from the user.

In addition, the method of providing a social network service using augmented reality according to an embodiment of the present application is a method of providing a social network service using augmented reality by the apparatus for providing a social network service using augmented reality according to the embodiment of the present application as described above. , (a) generating a user avatar corresponding to the user and storing the generated user avatar in a first storage unit; (b) overlapping and displaying the user avatar on a real image displayed on a screen of a user terminal in response to a shooting target area when it is detected that the shooting mode is in the shooting mode; (c) obtaining an augmented reality image in which the real image and the user avatar are combined and storing the augmented reality image in a second storage unit when a user input is made to the shooting button in the shooting mode; and (d) when a share request signal for sharing the augmented reality image with other users is input from the user, uploading the augmented reality image to an SNS relational network for sharing.

The above-described problem solving means are merely exemplary, and should not be construed as limiting the present application. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and detailed description.

According to the above-described problem solving means of the present application, by providing an apparatus and method for providing a social network service using augmented reality, problems with conventional SNS (eg, inconvenient to use SNS due to various advertisements and invasion of personal privacy) Problems, problems in the form of photos, limitations in expressing information using everyday sharing methods limited to short videos, problems that do not fully satisfy their needs for some people with a desire to show off, problems that pass without an element of interest It is possible to solve the problem that users gradually lose interest, etc.).

However, the effects obtainable herein are not limited to the above-described effects, and other effects may exist.

1 is a block diagram showing a schematic configuration of a system for providing a social network service using augmented reality according to an embodiment of the present application.
2 is a diagram illustrating an example of a first augmented reality image stored in a second storage unit in the apparatus for providing a social network service using augmented reality according to an embodiment of the present application.
3 is a diagram illustrating an example of a second augmented reality image stored in a second storage unit in the apparatus for providing a social network service using augmented reality according to an embodiment of the present application.
4 is a view for explaining a non-combination condition considered when acquiring an augmented reality image in the apparatus for providing a social network service using augmented reality according to an embodiment of the present application.
5 is a diagram schematically illustrating the configuration of an avatar manager in the apparatus for providing a social network service using augmented reality according to an embodiment of the present application.
6 is a schematic block diagram for explaining a simulation unit included in the apparatus for providing a social network service using augmented reality according to an embodiment of the present application.
7 is a view for explaining an example in which the region of the tightening portion identified by the identification unit in the simulation unit of the apparatus for providing a social network service using augmented reality according to an embodiment of the present application is displayed on the screen.
8 is a diagram illustrating an example of sensory data for a fashion product provided by the front display device 3 interlocked with the apparatus for providing a social network service using augmented reality according to an embodiment of the present application.
9 is an operation flowchart for a method of providing a social network service using augmented reality according to an embodiment of the present application.

Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art to which the present application pertains can easily carry out. However, the present application may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present application in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout this specification, when a part is "connected" with another part, it is not only "directly connected" but also "electrically connected" or "indirectly connected" with another element interposed therebetween. "Including cases where

Throughout this specification, when a member is positioned “on”, “on”, “on”, “on”, “under”, “under”, or “under” another member, this means that a member is positioned on the other member. It includes not only the case where they are in contact, but also the case where another member exists between two members.

Throughout this specification, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

1 is a block diagram showing a schematic configuration of a social network service (SNS, Social Network Services) providing system 1000 using augmented reality according to an embodiment of the present application.

The system 1000 for providing a social network service (SNS, Social Network Services) using augmented reality according to an embodiment of the present application provides a social network service (SNS, Social Network Services) using augmented reality according to an embodiment of the present application device 100 may be included.

Hereinafter, for convenience of description, the social network service providing system 1000 using augmented reality according to an embodiment of the present application is referred to as the present system 1000 , and an apparatus for providing social network service using augmented reality according to an embodiment of the present application (100) will be referred to as the present device (100). Also, a social network service (SNS) may be expressed differently as social media or the like.

Referring to FIG. 1 , the present system 1000 may include the present apparatus 100 and a plurality of user terminals 1 , 1 ′, ... .

The apparatus 100 generates a user avatar corresponding to the user, stores it in the first storage unit, and when it is detected that the shooting mode is in the shooting mode, the user is displayed on a real image displayed on the screen of the user terminal corresponding to the shooting target area. The avatar is displayed overlapped, and when a user input is made to the shooting button in the shooting mode, an augmented reality image in which the real image and the user avatar are combined is acquired and stored in the second storage unit, and the augmented reality image is shared with other users When a share request signal for sharing is input from the user, the augmented reality image may be uploaded to the SNS relational network for sharing. A detailed description of the apparatus 100 will be described later.

The plurality of user terminals 1, 1', ... may include a first user terminal 1 that is a user terminal of a first user, a second user terminal 1' that is a user terminal of a second user, and the like. However, this is only an example for helping understanding of the present application, and is not limited thereto, and the number of a plurality of user terminals considered herein may be set in various ways.

Each of the plurality of user terminals 1 , 1 ′, ... and the apparatus 100 may be connected through the network 2 . The device 100 may be linked (connected) with the user terminal 1 .

Hereinafter, in the description of the present application, the first user terminal 1 among the plurality of user terminals 1, 1', ... will be described based on the description of the user terminal. That is, in the following description, it is assumed that the user terminal is the first user terminal 1 . This is only an example to help the understanding of the present application, and is not limited thereto, and the description of the user terminal 1 (the first user terminal) is omitted even if the contents are omitted below. ', ...) can be equally applied to the description of each.

In addition, in the description of the present application, the description of the user will be based on the first user among a plurality of users. That is, in the following description, it is assumed that the user is the first user. This is only an example for helping understanding of the present application, and is not limited thereto, and the description of the user (the first user) may be equally applied to the description of each of a plurality of users, even if omitted below. have.

The device 100 may be linked (connected) with the user terminal 1 (the first user terminal). Here, when the device 100 is interlocked (connected) with the user terminal 1, the method of providing a social network service using augmented reality by the device 100 is a program executable in the user terminal 1 or When it can be implemented in the form of an application, it may mean a case where operation control is possible (state) by the user terminal 1 as it is installed in the user terminal 1 .

As another example, when the device 100 interworks with the user terminal 1 is a case in which the device 100 is provided in a configuration separate from the user terminal 1, the device 100 As the user terminal 1 is connected through the network 2 , it may mean a case (state) in which operation can be controlled remotely by the user terminal 1 .

In other words, the device 100 may be provided in the form of a device included in the user terminal 1 , for example. In this case, the device 100 may be a device that can be implemented in the form of a program or application installed in the user terminal 1 . In other words, the method for providing a social network service using augmented reality provided through the present device 100 may be implemented in the form of a program or application, for example, and provided through the user terminal 1 . However, the present invention is not limited thereto, and as another example, the device 100 may be provided in the form of a server or a separate device capable of transmitting and receiving data with the user terminal 1 .

A user terminal (ie, each of a plurality of user terminals) considered herein is, for example, a Personal Communication System (PCS), a Global System for Mobile communication (GSM), a Personal Digital Cellular (PDC), a Personal Handyphone System (PHS), and a PDA. (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (WCode Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smart phone It may include all types of wired/wireless communication devices such as a smart pad, a tablet PC, a notebook computer, a wearable device, and a desktop PC, but is not limited thereto.

The network 2 is, for example, a 3rd Generation Partnership Project (3GPP) network, a Long Term Evolution (LTE) network, a World Interoperability for Microwave Access (WIMAX) network, the Internet, a Local Area Network (LAN), and a Wireless LAN (Wireless LAN) network. Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), Bluetooth (Bluetooth) network, NFC (Near Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc. Types of wired/wireless networks may be included, but are not limited thereto.

The user terminal 1 may interwork with the apparatus 100 to receive a control signal for controlling the operation of the apparatus 100 from a user (the first user) and provide it to the apparatus 100 . The user terminal 1 may control (remotely control) various operations of the apparatus 100 . Hereinafter, the apparatus 100 will be described in more detail.

The device 100 may include an avatar management unit 210 , a screen control unit 220 , an image management unit 230 , an SNS management unit 240 , a first storage unit 250 , and a second storage unit 260 . . The avatar manager 210 may include an information acquisition unit 211 .

The avatar manager 210 may generate a user avatar corresponding to the user and store it in the first storage unit 250 . Here, the user may be the first user, and the user avatar of the first user may be the first user avatar.

In this case, in the example of the present application, the user is exemplified as the first user, but the present disclosure is not limited thereto, and not only the first user but also each of a plurality of users may be considered as the user. That is, the description of the user (the first user) may be equally applied to the description of each of the plurality of users, even if omitted below.

The avatar manager 210 may generate a user avatar (first user avatar) in consideration of body characteristic information received from the user (first user). The user avatar (first user avatar) generated by the avatar manager 210 may be a 3D avatar generated in consideration of at least one of a user's body type, face shape, facial expression, and skin tone received as body characteristic information. . That is, the first user avatar generated in consideration of the body characteristic information of the first user may be a 3D avatar generated in consideration of at least one of the body type, face shape, facial expression, and skin tone of the first user.

In this case, the description of the user avatar (the first user avatar) may be equally applied to the description of each of the plurality of user avatars, even if omitted below.

That is, the avatar manager 210 may generate a first user avatar in consideration of the body characteristic information received from the first user and store it in the first storage unit 250 . Also, the avatar manager 210 may generate a second user avatar in consideration of body characteristic information received from the second user and store the generated second user avatar in the first storage unit 250 .

In this case, in an example of the present application, both the first user avatar generated in response to the first user and the second user avatar generated in response to the second user are stored in the first storage unit 250 , which is a single storage unit. Although illustrated, the present invention is not limited thereto. As another example, the plurality of user avatars generated corresponding to each of the plurality of users may be stored in each of the plurality of sub storage units provided in the number corresponding to each of the plurality of users. That is, although not shown in the drawings, the first storage unit 250 includes a plurality of sub storage units, and may include a first sub storage unit, a second sub storage unit, and the like. In this case, the first sub storage unit is a storage unit provided in response to the first user, and may store a user avatar (first user avatar) generated in association with the first user. The second sub storage unit is a storage unit provided to correspond to the second user, and may store a user avatar (second user avatar) generated in association with the second user.

In addition, the description of the first storage unit 250 may be applied to the description of the second storage unit 260 to be described later in the same or similar manner. That is, the second storage unit 260 is an augmented reality image (first augmented reality image) acquired by the first user terminal 1 and the augmented reality image (second augmented reality images).

At this time, in an example of the present application, the first augmented reality image obtained through the first user terminal 1 by the first user and the second augmented reality obtained through the second user terminal 1 ′ by the second user Although it has been exemplified that the images are all stored in the second storage unit 260 which is a single storage unit, the present invention is not limited thereto. As another example, a plurality of augmented reality images obtained through each of the plurality of user terminals (1, 1', ...) are provided in a number corresponding to each of the plurality of user terminals (1, 1', ...) in a plurality of sub-storage may be stored in each section. That is, although not shown in the drawings, the second storage unit 260 is a plurality of lower storage units, and may include a first lower storage unit, a second lower storage unit, and the like. In this case, the first sub storage unit is a storage unit provided in response to the first user terminal (or the first user), and the first augmented reality image obtained through the first user terminal may be stored. The second lower storage unit is a storage unit provided in response to the second user terminal (or the second user), and may store the second augmented reality image obtained through the second user terminal.

As such, the first sub storage unit and the first sub storage unit are storage units provided in response to the first user (or the first user terminal), and may mean a first personal storage unit for the first user. In addition, the second sub storage unit and the second sub storage unit are storage units provided in response to the second user (or the second user terminal), and may mean a second personal storage unit for the second user.

The screen controller 220 may overlap and display the user avatar (the first user avatar) on the real image displayed on the screen of the user terminal 1 in response to the shooting target area when it is detected that the shooting mode is in the shooting mode. .

Here, the photographing target area refers to an area photographed through the image sensor of the first user terminal 1 (a photographing area capable of being photographed through the image sensor), and for example, a specific place, object (person, object, etc.) , may mean a photographing area related to a landscape or the like. However, the present invention is not limited thereto, and any area that can be photographed through the image sensor of the first user terminal 1 may be applied to the photographing target area.

Also, the photographing mode may mean a mode for the first user to photograph the photographing target area through the image sensor of the first user terminal 1 .

When the screen controller 220 overlaps and displays the user avatar (the first user avatar) on the real image displayed on the screen of the user terminal 1 , for example, the first user avatar is input by the first user on the real image. It can be displayed in a setting location based on user input that is set based on . According to this, as an example, the first user freely moves (changes) the first user avatar displayed on the real image by the screen controller 220 on the screen of the first user terminal 1 by dragging, touching, or the like. By doing so, the display position of the first user avatar on the real image (ie, the position where the first user avatar is placed or displayed on the real image) may be set.

Exemplarily, the first user may determine the location of the first user avatar on the real image 51 displayed on the screen of the first user terminal 1, for example, the first location 71 shown in FIG. The second position 72 and the third position 73 can be freely moved and changed/set respectively. For example, when the first user input related to setting the display position of the first user avatar is made with respect to the third location 73 among the plurality of locations 71 , 72 , and 73 , the third location where the first user input is made Recognizing that 73 is the user input-based setting position, the screen controller 220 may display the first user avatar in the third position 72 which is the user input-based setting position.

However, the present invention is not limited thereto, and as another example, when the screen controller 220 displays a real image on the screen of the user terminal (first user terminal, 1) for a preset time (for example, 3 seconds) , Recognizes this as a signal for receiving a recommendation for the location of the first user avatar to be displayed on the real image (that is, related to the avatar display requesting the recommendation of a location determined to be most suitable for displaying the first user avatar on the real image) Recognized as a location recommendation request signal), at least one previously selected recommended location is displayed on the real image, and the user avatar (first selected and recommended) is located at any one of the at least one previously selected recommended location displayed on the real image. 1 user avatar) can be positioned and displayed. Here, any one of the at least one previously selected and recommended location may be a location selected by the input of the first user.

In addition, the at least one preselected recommended location is a location selected in consideration of object information identified through image analysis of the real image 51 displayed on the screen of the first user terminal 1 and/or in the SNS network. Selected in consideration of statistical location information related to the display position of the user avatar calculated based on analysis of a plurality of uploaded augmented reality images (that is, a plurality of augmented reality images uploaded by multiple users and stored in the second storage unit) It may be a location where

Here, the object information identified through image analysis of the real image 51 is information about an object included in the real image 51, and such object information includes, for example, a chair, a person, an animal, an entrance to a building, It may include a sculpture, a tree, and the like, but is not limited thereto. In this case, various image analysis algorithms known in the prior art or developed in the future may be applied to image analysis.

For example, when the entrance and chair of the building are identified as object information through image analysis of the real image 51 , at least one pre-selected recommended location is the second corresponding to the identified entrance of the building in the real image 51 . 1 position and a position corresponding to the chair identified in the real image 51 may be included. At this time, when the real image 51 is displayed on the screen of the first user terminal 1 for a preset time, the screen controller 220 displays the first position and the second position on the real image 51, at this time , when a user input for the second location is made, the first user avatar may be displayed at the second location on the real image 51 . Accordingly, the screen controller 220 may display the first user avatar on the real image 51 to cover the chair included in the real image 51 .

In this case, the screen controller 220 may change the posture of the first user avatar according to the type of object corresponding to any one preselected recommended position selected by the user input as needed, and change the posture of the first user avatar to the user. It can be displayed at any one previously selected recommended position selected by input. For example, it is assumed that a second location is selected by a user (first user) input as one previously selected recommended location among a plurality of previously selected and recommended locations including a first location and a second location.

In this case, the screen controller 220 sets the posture of the first user avatar as the posture of the basic first user avatar because the object corresponding to any one preselected recommended position (second position) selected by the user input is a chair. A posture in which the first user avatar sits in a chair may be changed from the attentive posture to a sitting posture in a chair. Thereafter, the screen controller 220 may display the posture-changed first user avatar (ie, the first user avatar having a sitting posture in a chair) at a second position (a position corresponding to the chair) on the real image 51 . .

As another example, the screen controller 220 may include a plurality of augmented reality images uploaded to an SNS relational network (ie, a plurality of augmented reality images uploaded by a plurality of users and stored in the second storage unit) in order to select at least one previously selected recommended location. of the augmented reality image) (in particular, a plurality of augmented reality images related to the real image 51 uploaded in relation to the real image 51) can be analyzed, and a plurality of augmented reality images related to the real image 51 can be analyzed through the analysis. Calculation of statistical information (ie, statistical location information, which is statistical information about the positions at which user avatars are displayed in a plurality of augmented reality images related to real images) on which positions of the user avatars included in the real image 51 appear can do.

At this time, as a result of analyzing the positions (display positions) of user avatars (hereinafter referred to as all user avatars) in a plurality of augmented reality images related to the real image 51 pre-stored in the second storage unit 260 by a plurality of users, For example, with respect to the plurality of locations 71 , 72 , and 73 shown in FIG. 2 , user avatars corresponding to 70% of all user avatars are displayed in the first location 71 , and 20 of all user avatars are displayed in the first location 71 . % of the user avatars are displayed at the third location 73 , and user avatars corresponding to 9% of the total user avatars are displayed at the second location 72 , corresponding to 1% of the total user avatars. It is assumed that user avatars are displayed at a fourth location (not shown).

In this case, the screen control unit 220 controls 70% of the first position 71, 10% of the second position 72, 20% of the third position 73, and 1% of the fourth position (not shown). It is possible to calculate statistical location information including Thereafter, as an example, the screen control unit 220 sets the second position as a position belonging to a preset high number (eg, top 3) among a plurality of positions (eg, first to fourth positions) corresponding to the calculated statistical position information. The first position 71 to the third position 73 may be displayed on the real image 51 as at least one previously selected recommended position. Thereafter, for example, when a user input is made to a third location 73 as one of at least one previously selected and recommended location on the real image 51 , the first user avatar is displayed on the real image 51 . ) can be displayed at the third position 73 on the

As another example, when the statistical location information is calculated, the screen controller 220 displays the most user avatars among a plurality of locations corresponding to the calculated statistical location information (user avatars corresponding to 70% of all user avatars are The first position 71, which is the highest position identified as displayed) The selected recommended location may be determined, and then, the first user avatar may be displayed in the determined first location 71 which is any one previously selected and recommended location.

After the user avatar is displayed on the real image by the screen controller 220 , the image management unit 230 displays the real image and the user in response to the user input when the first user's user input for the shooting button is made in the shooting mode. The augmented reality image combined with the avatar (the first user avatar) may be acquired and stored in the second storage unit 260 .

In this case, for example, the shooting button may be provided in a touch input method on an area of the screen of the first user terminal 1 , or may be provided in a physical button method on the first user terminal 1 , limited thereto. it is not going to be

The first user, under the control of the screen control unit 220 , the user avatar (the first user avatar) overlapped and displayed on the real image displayed on the screen of the user terminal 1, and then a photographed image (augmented reality image) ), it is possible to perform an input to the photographing button of the first user terminal 1 . As such, when the first user's user input (first user input) for the shooting button is made, the image management unit 230 responds to the first user input with the real image displayed on the first user terminal 1 and the second user input. An augmented reality image (a first augmented reality image) combined with a user avatar may be acquired and stored in the second storage unit 260 (particularly, stored in the first lower storage unit within the second storage unit).

The augmented reality image obtained through the first user terminal 1 means an augmented reality image in which a real image and a first user avatar are combined, which may be referred to as a first augmented reality image.

An example of the first augmented reality image stored in the second storage unit 260 may be more easily understood with reference to FIG. 2 .

2 is a diagram illustrating an example of the first augmented reality image 53 stored in the second storage unit 260 in the apparatus 100 for providing a social network service using augmented reality according to an embodiment of the present application. Here, the first augmented reality image 53 may mean an augmented reality image acquired through the first user terminal 1 .

Referring to FIG. 2 , it is assumed that a first user intends to take a picture using a hanok in a palace as an example by using the first user terminal 1 . At this time, when it is detected that the first user terminal 1 is in the shooting mode, the screen controller 220 displays the real image 51 corresponding to the shooting target area on the screen of the user terminal 1, and together with the avatar The first user avatar 52 generated and stored in the management unit 210 may be overlapped on the real image 51 and displayed (expressed) on the screen of the first user terminal 1 .

That is, when it is detected that the first user terminal 1 is in the photographing mode, the screen controller 220 may display the real image 51 displayed on the screen of the first user terminal 1 in response to the photographing target area. The first user avatar 52 generated by the avatar manager 210 may be overlapped and displayed (expressed) on the screen of the first user terminal 1 .

Thereafter, when the first user's input (first user input) to the shooting button is made in the shooting mode, the image management unit 230 displays the real image 51 and the second image displayed on the screen of the first user terminal 1 . One user avatar 52 is combined with an augmented reality image 53 (a first augmented reality image) obtained from the first user terminal 1, and the second storage unit 260 (particularly, the first lower level in the second storage unit) storage) can be saved.

That is, the image management unit 230 generates a first augmented reality image 53 by combining the first user avatar 52 with respect to the real image 51 when the first user's input for the shooting mode is made, The generated first augmented reality image 53 may be acquired and stored in the second storage unit 260 .

Herein, the augmented reality image stored in the second storage unit 260 may mean an image (image, photo) in a static state, but is not limited thereto, and an image in a static state such as an image or a photo. In addition, it is preferable to be understood as a concept including all images (movies, etc.) in a dynamic state.

When a share request signal for sharing the augmented reality image with other users is input from the user, the SNS management unit 240 may upload the augmented reality image to the SNS network for sharing in response to the user input for the share request signal. .

That is, when a share request signal for sharing the first augmented reality image 53 stored in the second storage unit 260 to another user is input from the first user, the SNS management unit 240 responds to the share request signal. In response to the input (input of the first user), the first augmented reality image 53 may be uploaded to the SNS relational network for sharing.

Here, the SNS relational network is a relational network (network) provided by the device 100, which may mean a relational network (a relational network formed between people) between people who share a specific interest or activity.

Although not shown in the drawings, the apparatus 100 may include a controller (not shown). A control unit (not shown) may control the operation of each unit included in the apparatus 100 .

The first storage unit 250 may store the user avatar generated in response to the user as described above. For example, the first storage unit 250 may store the first user avatar generated in response to the first user.

The second storage unit 260 may store the augmented reality image obtained from the user terminal as described above. For example, the second storage unit 260 may store the first augmented reality image 53 obtained from the first user terminal 1 .

On the other hand, after the first augmented reality image 53 is stored in the second storage unit 260 , the second user uses the second user's user terminal (the second user terminal, 1'), and the first user The same shooting target area (same shooting target) as the shooting target area that was photographed (that is, the shooting target area where the first augmented reality image 53 was obtained, which means the shooting target area corresponding to the first augmented reality image) Let's say you want to shoot a region).

At this time, it is determined whether the photographing target area photographed using the second user terminal 1' and the photographing target area photographed by the first user terminal 1 are the same photographing target area (that is, the second user terminal Determination of whether the shooting mode is for shooting the same shooting target area) is, for example, the first augmented reality image (corresponding to the shooting target area photographed by the first user terminal) pre-stored in the second storage unit 260 . The first location and orientation information (ie, the first location and orientation information including the location information and orientation information of the first augmented reality image) of the first augmented reality image) and the photographing displayed on the screen of the second user terminal 1 ' The comparison may be made between the second position and orientation information of the real image corresponding to the target region (second position and orientation information including the position information and orientation information of the real image displayed on the screen of the second user terminal).

For example, when the second location and orientation information is located within a preset error range from the first location and orientation information, the screen controller 220 may control the area to be photographed using the second user terminal 1 ′ and the first user. It may be determined that the photographing target areas photographed by the terminal 1 are the same photographing target areas.

According to this, when the image management unit 230 stores the augmented reality image in the second storage unit 260 , the augmented reality image, the user avatar included in the augmented reality image, location information about the location where the augmented reality image is obtained, and augmented reality The azimuth information regarding the azimuth from which the real image is acquired, the non-combination condition described below preset for the user avatar, etc. may be linked together and stored.

The screen control unit 220, the second user's user terminal (second user terminal, 1 ') is obtained by the first user terminal (1), the first augmented reality image 53 stored in the second storage unit (260) The first user avatar as a pre-stored user avatar in relation to the shooting target area (same shooting target area) is detected as being in a shooting mode for shooting the same shooting target area (same shooting target area) as the shooting target area corresponding to ). If 52 exists, the second user avatar and the first user avatar may be overlapped and displayed on the real image displayed on the screen of the user terminal 1 ′ of the second user. This can be more easily understood with reference to FIG. 3 .

3 is a diagram illustrating an example of a second augmented reality image 63 stored in the second storage unit 260 in the apparatus 100 for providing a social network service using augmented reality according to an embodiment of the present application. Here, the second augmented reality image 63 may mean an augmented reality image obtained through the second user terminal 1 ′.

Referring to FIG. 3 , as described above, after the first augmented reality image 53 is stored in the second storage unit 260 , the second user uses the second user terminal 1 ′ to perform the first augmentation. It is assumed that a photographing target area (same photographing target area) that is the same as the photographing target area corresponding to the real image 53 is to be photographed.

At this time, when it is detected that the second user terminal 1 ′ is in the shooting mode, the screen controller 220 , corresponds to the shooting mode among a plurality of augmented reality images pre-stored in the second storage unit 260 , the second user Whether an augmented reality image targeting the same shooting target area (same shooting target area) as the shooting target area corresponding to the real image 61 displayed on the screen of the terminal 1' exists in the second storage unit 260 can judge Here, the augmented reality image pre-stored in the second storage unit 260 having the same shooting target region as the shooting target region corresponding to the real image displayed on the screen of the second user terminal 1 ′ is augmented matching in the description to be described later. It may be referred to as a real image, and in this case, let the matching augmented reality image be the first augmented reality image 53 as an example.

In this case, the screen controller 220, when the second user terminal 1' is in the shooting mode, the shooting target area corresponding to the real image displayed on the screen of the second user terminal 1' is the same as the shooting target area When the augmented reality image (ie, the first augmented reality image that is the matching augmented reality image) for the (same shooting target area) exists in the second storage unit 260, the shooting mode of the second user terminal 1' of at least one augmented reality image (for example, a first augmented reality image that is a matching augmented reality image) among a plurality of augmented reality images stored in the second storage unit 260 in the same shooting target area as the shooting target area (same shooting) It can be recognized as a shooting mode for shooting the target area).

Then, the screen control unit 220, the second user's user terminal (1 ') at least one of the plurality of augmented reality images pre-stored in the second storage unit 260 (eg, matching augmented reality image) It is detected that the shooting mode is for shooting the same shooting target region (same shooting target region) as the shooting target region of the first augmented reality image), but as a pre-stored user avatar in relation to the shooting target region (same shooting target region) Assume that a first user avatar exists. That is, in the second storage unit 260 , the augmented reality image (matching augmentation) obtained and stored by photographing the same shooting target area (same shooting target area) as the shooting target area that the second user's terminal 1 ′ wants to shoot. Assume that a first augmented reality image 53 exists as a real image), and a first user avatar 52 exists as a pre-stored user avatar in relation to the first augmented reality image 53 .

In this case, the screen control unit 220 may include a second user avatar 62 that is a user avatar generated in response to the second user on the real image 61 displayed on the screen of the user terminal 1 ′ of the second user. can be displayed overlappingly, and in addition to this, augmented reality previously stored in the second storage unit 260 in relation to the real image 61 displayed on the screen of the user terminal 1' of the second user. The first user avatar 52, which is the user avatar in the image (the first augmented reality image that is the matching augmented reality image), is overlapped and displayed on the real image 61 displayed on the screen of the user terminal 1' of the second user. can do.

That is, in relation to the real image 61 displayed on the screen of the second user terminal 1 ′, the screen control unit 220 controls the first user avatar 52 previously stored in the second storage unit 260 . and the second user avatar 62 may be overlapped and displayed on the real image 61 displayed on the screen of the second user terminal 1'.

Thereafter, the image management unit 230 may, for example, when a user input (a second user's input) is made with respect to a shooting button displayed on a screen area of the second user terminal 1', the second user terminal 1' As an augmented reality image in which all of the real image 61, the second user avatar 62, and the first user avatar 52 displayed on the screen of are combined, the second augmented reality image 63 is transferred to the second user terminal 1' ), and may store the obtained second augmented reality image 63 in the second storage unit 260 (in particular, store it in a second lower storage unit within the second storage unit).

At this time, the image manager 230 acquires and stores the second augmented reality image 63 in which the real image 61, the second user avatar 62, and the first user avatar 52 are combined, the first When a non-combination condition is previously set in relation to the user avatar 52 , combining for acquiring the second augmented reality image 63 may be performed in consideration of the non-combination condition. The image manager 230 may determine whether to perform combining for obtaining the second augmented reality image 63 and/or to acquire the second augmented reality image 63 in consideration of the uncombinable condition.

That is, when the second user's input for the shooting mode is made, the image manager 230 combines the second user avatar 62 and the first user avatar 52 with respect to the real image 61 to thereby create the second augmented reality. An image 63 may be generated, and the generated second augmented reality image 63 may be acquired and stored in the second storage unit 260 .

At this time, the image management unit 230, when trying to combine the second user avatar 62 and the first user avatar 52 with the real image 61 to obtain the second augmented reality image 63, When a non-combination condition is previously set in relation to the first user avatar 52 , combining for acquiring the second augmented reality image 63 may be performed in consideration of the non-combination condition.

That is, when the image manager 230 acquires the second augmented reality image 63 through the second user terminal 1 ′, the pre-stored user avatar (eg, the first A second augmented reality image 63 in which a plurality of user avatars including a user avatar) and a current user avatar (for example, a second user avatar) corresponding to the user terminal detected as being in the current shooting mode are combined with a real image In the case of acquisition, if a non-combination condition is previously set in relation to the pre-stored user avatar, combining (combining a plurality of user avatars with respect to the real image) for the acquisition of the second augmented reality image 63 in consideration of the uncombinable condition ) can be done. This can be more easily understood with reference to FIG. 4 .

4 is a view for explaining a non-combination condition considered when acquiring an augmented reality image in the apparatus 100 for providing a social network service using augmented reality according to an embodiment of the present application.

Referring to FIG. 4 , the image manager 230 combines the second user avatar 62 and the first user avatar 52 with the real image 61 to obtain the second augmented reality image 63 . At this time, when a non-combination condition is previously set in relation to the first user avatar 52 , the second augmented reality image 63 is taken in consideration of the uncombinable condition set in relation to the first user avatar 52 . can be combined to obtain .

That is, the image management unit 230 may display the second user avatar 62 and the first user avatar 52 on the real image 61 in consideration of a previously set uncombinable condition in relation to the first user avatar 52 . ) may be combined to obtain the second augmented reality image 63 and store it in the second storage unit 260 .

Here, the non-binding condition may include a first condition and a second condition. The first condition is that when there is at least a partial overlapping area s between the first user avatar 52 and the second user avatar 62 that are overlapped and displayed on the real image 61, the user avatar in the forward and backward directions It may mean a condition set in relation to the location of The second condition is that when at least a part of the overlapping area s exists between the first user avatar 52 and the second user avatar 62 that are overlapped and displayed on the real image 61, the overlapping area s is It may mean a condition set in relation to a body part of the corresponding user avatar.

Specifically, the first condition is that when there is at least a partial overlapping area s between the first user avatar 52 and the second user avatar 62 that are overlapped and displayed on the real image 61 , the second condition is It may mean a condition that the user avatar 62 is located in front of the first user avatar 52 .

Exemplarily, when the second user's input to the capture button is made to obtain the second augmented reality image 63 (at the time when the second user's input is made), the second user terminal 1' On the real image 61 displayed on the screen, the first user avatar 52 and the second user avatar 62 are displayed to have at least a partial overlapping area s with each other, and the second user avatar ( Assume that the overlapped area of 62 is displayed to be located in front of the overlapped area of the first user avatar 52 (ie, the overlapped area of the first user avatar is obscured by the overlapped area of the second user avatar). Let's say there is). In this case, if the first user avatar 52 has a pre-set condition that cannot be combined with the first condition, the image manager 230 determines that the overlapping area of the second user avatar 62 is the first user avatar 52 . As it is located in front of the overlapping area of When an input is made, the second augmented reality image 63 may not be acquired in response to the second user's input (that is, the second augmented reality image may not be acquired).

If it is assumed that the overlapping area of the second user avatar 62 is displayed to be located behind the overlapping area of the first user avatar 52 (that is, the overlapping area of the second user avatar 62 is the overlapping area of the first user avatar). Let's say it is obscured by an overlapping region of ). In this case, if the first user avatar 52 has a pre-set condition that cannot be combined with the first condition, the image manager 230 determines that the overlapping area of the second user avatar 62 is the first user avatar 52 . As it is located behind the overlapping region of The second augmented reality image 63 may be acquired in response to the user's input and stored in the second storage unit 260 .

According to this, the fact that the first user avatar is set with the uncombinable condition related to the first condition means that another user (eg, the second user) is augmented with the first user's own avatar (the first user avatar). In allowing a real image to be acquired, the avatar of the other user (eg, the second user avatar) is positioned in front of the first user avatar so that at least a portion of the first user avatar is the avatar of the other user (the second user avatar). ) may mean a setting that restricts other users from acquiring the augmented reality image including the first user avatar.

When another user (eg, a second user, etc.) wants to acquire an augmented reality image including a first user avatar, the first user's avatar (the first user avatar) is the avatar of another user (eg, the first user avatar) 2 When it is desired that the shape (shape) of the first user avatar is completely displayed in the front of the augmented reality image without being obscured by the user avatar, etc.) You can set an impossible condition. That is, when the first user wants other users to obtain an augmented reality image including his/her avatar (the first user avatar), he/she wants to show the first user avatar in front of the augmented reality images acquired by other users. In this case, a non-combination condition related to the first condition may be set for the first user avatar.

On the other hand, the second condition is that at least a partial overlapping area (s) exists between the first user avatar 52 and the second user avatar 62 that are overlapped and displayed on the real image 61, but the overlapping area (s) ) has a size less than or equal to a preset size and the second user avatar 62 is positioned in front of the first user avatar 52, the second user avatar 62 corresponding to the overlapping area s is The body part may mean a condition corresponding to a preset body part.

Here, the preset body part may be, for example, a lip part, a hand part, etc., but is not limited thereto, and may be set by a user input.

Exemplarily, when the second user's input to the capture button is made to obtain the second augmented reality image 63 (at the time when the second user's input is made), the second user terminal 1' On the real image 61 displayed on the screen, the first user avatar 52 and the second user avatar 62 are displayed to have at least a partial overlapping area (s) with each other, the overlapping area (s) Assume that the size of has a size equal to or smaller than a preset size, and the overlapping area of the second user avatar 62 is displayed to be positioned in front of the overlapping area of the first user avatar 52 (that is, the first user Assume that the overlapping area of the avatar is obscured by the overlapping area of the second user's avatar).

In this case, the image manager 230 determines whether the body part of the second user avatar 62 (ie, the body part of the overlapping region of the second user avatar) corresponding to the overlapping region s having a size less than or equal to a preset size is selected. It may be determined whether it corresponds to a preset body part. In this case, the image manager 230 sets the body part of the second user avatar 62 corresponding to the overlapping region s (ie, the body part of the overlapping region of the second user avatar) as a preset body part, as an example, the lips. In the case of a part, the first user avatar 52 and the second user avatar 62 are interposed between the first user avatar 52 and the second user avatar 62 on the real image 61 displayed on the screen of the second user terminal 1'. ), it can be determined that the display (position, arrangement) is displayed in a state that satisfies the non-binding condition related to the preset second condition.

As such, the shape of the overlapping region s between the first user avatar 52 and the second user avatar 62 displayed on the real image 61 displayed on the screen of the second user terminal 1 ′ is the second condition. In the case of a form that satisfies the condition not to be combined with respect to (that is, when only the portion corresponding to the lips of the second user avatar is displayed to be positioned in front of the first user avatar while overlapping the first user avatar) , the image management unit 230 determines that the second augmented reality image 63 to be acquired satisfies the uncombinable condition related to the second condition, and as a result, when the second user's input to the capture button is made, the second 2 It is possible to prevent the acquisition of the second augmented reality image 63 in response to the user's input (that is, it is possible to prevent the second augmented reality image from being acquired.

If the size of the overlapping area s between the second user avatar 62 and the first user avatar 52 exceeds a preset size, the second user avatar 62 is When positioned in front of or behind the avatar 52 , the image management unit 230 performs the first user avatar 52 on the real image 61 displayed on the screen of the second user terminal 1 ′ and It is determined that the second user avatar 62 does not satisfy (does not satisfy) the condition for not being able to combine with respect to the second condition preset for the first user avatar 52 with each other. When the second user's input is made, the second augmented reality image 63 may be acquired in response to the second user's input and stored in the second storage unit 260 .

According to this, when the first user avatar 52 is set with a non-combination condition related to the second condition, another user (for example, the second user) has the first user's own avatar (the first user avatar) In allowing the included augmented reality image to be acquired, only a region corresponding to a preset body part (eg, lips, hand) among other user's avatars (eg, second user avatar) is greater than that of the first user's avatar. It may mean a setting that restricts other users from acquiring the augmented reality image including the first user avatar only in the case where it is located in front.

When another user (for example, a second user, etc.) tries to acquire an augmented reality image including the first user avatar, an act of harassing the first user avatar or a provocative act using a gesture (i.e., In the case of preventing the acquisition of an augmented reality image including an action that causes discomfort in relation to the first user avatar), the first user may set a non-combination condition related to the second condition with respect to the first user avatar. . That is, when the first user enables other users to acquire an augmented reality image including his/her avatar (the first user avatar), the first user is in a healthy state without any unpleasant behavior such as harassment or provocation in relation to the first user avatar. When it is desired to obtain an augmented reality image including the user avatar, a non-combination condition related to the second condition may be set for the first user avatar.

As an example, the avatar management unit 210 may generate a first user avatar corresponding to the first user and store it in the first storage unit 250 , and in this case, when receiving input from the first user, setting information related to the non-combinable condition, The setting information related to the non-combination condition received from the first user may be stored in the first storage unit 250 in association with the first user avatar. The setting information related to the non-combinable condition may include first condition setting information for determining whether the first condition is satisfied and second condition setting information for determining whether the second condition is satisfied.

If, in relation to the first user avatar, there is no setting information related to the non-combination condition (that is, when the non-combination condition is not previously set in relation to the first user avatar), the first user avatar and the other users A combination can be made between two user avatars without any restrictions between the avatars of .

In addition, as described above, the screen controller 220 stores the second user's user terminal (the second user terminal, 1') obtained by the first user terminal 1 and stored in the second storage unit 260 . It is detected that the shooting mode is for shooting the same shooting target area (same shooting target area) as the shooting target area corresponding to the first augmented reality image 53, but pre-stored in relation to the shooting target area (same shooting target area) When the first user avatar 52 exists as the user avatar, the second user avatar and the first user avatar may be overlapped and displayed on the real image displayed on the screen of the user terminal 1' of the second user. .

In this case, when the first user avatar is displayed on the real image displayed on the screen of the second user terminal 1 ′, the screen controller 220 is configured to perform the second user avatar based on preset disclosure range setting information for the first user avatar. 1 It is possible to determine whether to display the user avatar.

Here, the preset disclosure range setting information may refer to information that sets a range allowing the display of the first user avatar to terminals of other users other than the first user. All of the public scope setting information may include three types of setting information including public, friend public, and private. Such public range setting information may be set by a user input. For example, the disclosure range setting information for the first user avatar may be set by a first user input.

For example, suppose that the second user's terminal 1' is detected to be in a shooting mode for shooting the same shooting target area, but a first user avatar exists as a pre-stored user avatar in relation to the same shooting target area. In this case, the screen controller 220 performs the second user avatar 62 and the first user avatar 52 on the real image 61 displayed on the screen of the second user terminal 1 ′ before displaying the second user avatar 62 and the first user avatar 52 . 1 It may be checked whether the disclosure range setting information for the user avatar 52 exists.

At this time, if 'private' information exists as preset disclosure range setting information in the first user avatar, the screen control unit 220 displays the information on the real image 61 displayed on the screen of the second user terminal 1 '. Only 2 user avatars 62 can be displayed.

As another example, when the information 'friend public' exists as preset disclosure range setting information in the first user avatar, the screen control unit 220 controls the first user and the second user corresponding to the first user avatar 52 . A real image displayed on the screen of the second user terminal 1' if the second user corresponding to the user avatar 62 is in a friending state with each other in the device 100, and if the confirming result is in the friending state The second user avatar 62 and the first user avatar 52 may be displayed together on 61 . On the other hand, if the confirmation result is not in the friending state, the screen controller 220 displays only the second user avatar 62 on the real image 61 displayed on the screen of the second user terminal 1', and the first user The avatar 52 may not be displayed.

As another example, if 'public all' information exists as preset disclosure range setting information in the first user avatar, the screen control unit 220 controls the first user and the second user corresponding to the first user avatar 52 . 2 On the real image 61 displayed on the screen of the second user terminal 1', regardless of whether the second user corresponding to the user avatar 62 is in a state of making friends with each other in the device 100 Both the second user avatar 62 and the first user avatar 52 may be displayed.

As an example, it is assumed that the first user is a celebrity, preset public range setting information for the first user avatar 52 is set to 'public all', and the second user is a fan of the first user. In this case, the second user receives the second augmented reality image 63 in which the first user avatar 52 and the second user avatar 62 are combined on the real image 61 as an augmented reality image through the device 100 . ) may be obtained and stored in the second storage unit 260 . According to this, as the user avatars displayed on the screen of the user terminal are 3D avatars generated in consideration of the body characteristic information of each user in order to acquire the augmented reality image, the second user is actually the first user in the photographing target area. Despite not meeting the user and taking a picture with the first user, the second augmented reality image 63 provided through the device 100 makes it seem as if the second user took a picture with the first user in the area to be photographed. You can provide it so that you can get the same feeling as when you took it.

Meanwhile, the avatar manager 210 may include an information acquisition unit 211 . When it is detected that the user terminal (the first user terminal, 1 ) of the first user is in the shooting mode, the information obtaining unit 211 may be configured to detect a user (first user) corresponding to the shooting mode detection time, which is the sensing time to be in the shooting mode. ) of biometric information, weather forecast information corresponding to the detection time of the shooting mode, and upload information uploaded by the user (the first user) may be analyzed to obtain at least one of the calculated upload information analysis result information.

In this case, the biometric information of the user (the first user) may include, but is not limited to, the user's blood pressure, body temperature, pulse, respiration rate, ECG signal, and the like. The user's biometric information may be, for example, wearable on a body part of the user (the first user), and may be obtained from a wearable device connected to the apparatus 100 and the network 2 through a wired/wireless connection. In this case, the wearable device may refer to a type of device that can be worn on a wrist, head, neck, etc. as a part of the user's body. The wearable device may include, for example, a smart watch and the like, but is not limited thereto, and may be applied in any form as long as it is wearable on a part of the user's body.

The weather forecast information may be obtained from a server (not shown) of the Korea Meteorological Administration, for example, connected to the device 100 and the network 2 through the network 2 .

The upload information analysis result obtained by the information acquisition unit 211 may be, for example, information calculated by analyzing at least one piece of upload information uploaded by the user (the first user) to the SNS relational network. In this case, the upload information analysis result may mean, for example, information analyzed in relation to the emotional state of the user (the first user) (ie, emotional state-related analysis information), but is not limited thereto. Here, the emotional state may include, but is not limited to, a joy state, a sad state, a depressed state, and the like.

In addition, the upload information analysis result, for example, may be information calculated by analyzing previously uploaded upload information on a date corresponding to the detection time of the shooting mode, but is not limited thereto, and within a period preset by the user (eg, For example, it may be information calculated by analyzing previously uploaded upload information (a period corresponding to three days before the date corresponding to the detection time of the shooting mode).

In this case, the upload information considered when calculating the upload information analysis result may include various types of information uploaded in relation to blogs, postings, and posts, and may include postings (contents) related to blogs, postings, posts, and the like. In addition, the upload information may include message information transmitted and received in the user terminal (the first user terminal), but is not limited thereto.

The avatar manager 210 may be configured to generate a user avatar generated by the avatar manager 210 based on at least one piece of information acquired by the information acquirer 211 (ie, a first user generated by taking body characteristic information into consideration). The state information of the avatar) may be updated, and an updated user avatar (updated first user avatar) may be generated and stored in the first storage unit 250 . Thereafter, when it is detected that the first user terminal 1 is in the shooting mode, the screen controller 220 is configured to control the screen of the first user terminal 1 based on the updated user avatar (updated first user avatar). An updated user avatar (updated first user avatar) may be displayed on the real image displayed in the .

Here, the state information of the user avatar (the first user avatar) may include at least one of a facial expression, a posture (action), a comment, and wearable clothes (wearing clothes that are clothes worn on the user avatar) of the user avatar.

The state information of the user avatar (the first user avatar) is automatically obtained based on at least one piece of information acquired by the information acquisition unit 211 .

For example, the avatar manager 210 may generate a first user avatar (which may be referred to as a basic first user avatar) in consideration of the body characteristic information of the first user. At this time, as a result of analyzing the at least one piece of information acquired by the information acquisition unit 211 , it is determined that the user's biometric information obtained in response to the detection time of the shooting mode is determined to be within the normal level range, for example, and the weather forecast information corresponds to sunny weather, and when the uploaded information is analyzed on the date corresponding to the detection time of the shooting mode and the result of analyzing the uploaded information is analyzed to be in a state of joy, based on the result of comprehensive analysis of these, the avatar management unit 210 ) may be determined to be a joy state as the current state of the user predicted in response to the sensing time of the shooting mode.

That is, the avatar management unit 210 determines whether the user (the first one), based on at least one piece of information acquired by the information obtaining unit 211 (by comprehensively analyzing at least one piece of information) in response to the time point at which the shooting mode is sensed. It is possible to determine the predicted current state information of the user, which is information that predicts the current state of the user) (the current state of the user corresponding to the time of sensing the shooting mode). Thereafter, as it is determined that the predicted user's current state is a joy state, the avatar manager 210 determines that the first user avatar (basic first user avatar) has a face as state information of the first user avatar (basic first user avatar) generated in consideration of body characteristic information. You can update your expression

For example, the basic facial expression of the first user avatar may be an expressionless expression, and the updated facial expression of the first user avatar may be a smiling facial expression. The avatar manager 210 may generate an updated first user avatar and store it in the first storage unit 250 . Alternatively, the avatar manager 210 may change the first user avatar (basic first user avatar) pre-stored in the first storage unit 250 into the updated first user avatar. In addition, the avatar management unit 210 generates an updated first user avatar and stores it in the first storage unit 250 , while the screen control unit 220 controls the first user terminal 1 in response to sensing the shooting mode. The updated first user avatar may be displayed on the real image displayed on the screen.

At this time, when the first user's input to the shooting button is made in the shooting mode, the image management unit 230 acquires an augmented reality image in which the real image and the updated first user avatar are combined as an augmented reality image, and stores the second. may be stored in the unit 260 .

As such, the apparatus 100 predicts ( determining the predicted user's current status information), reflecting the predicted user's current status, and taking the body characteristic information into consideration in advance of the first user avatar (basic first user avatar) status information (facial expression, posture, Wearing clothing, comments, etc.) and displaying the updated first user avatar on the screen, when the augmented reality image is acquired, the augmented reality image including the updated first user avatar reflecting the user's current state is displayed. It can be provided for the acquisition to take place.

This device 100 receives the augmented reality image reflecting the user's current emotion more through the acquisition of the augmented reality image reflecting the avatar (updated first user avatar) reflecting the user's current state (mood, etc.) It can be provided so that it can be shared with

That is, the device 100 allows the user avatar (the first user avatar) expressed on behalf of the first user, not the actual appearance, to be included in the acquired augmented reality image (especially, an update reflecting the user's current emotions) to be included in the augmented reality image), and the augmented reality image thus obtained can be shared with others. This device 100 has problems with conventional SNS (for example, there is a risk of exposure of personal life, and there is a limit in expressing information because it uses a daily sharing method limited to photo format and short video. , for some people who have a desire to show off, they do not fully satisfy their needs, and users lose interest over time, etc.) can be solved.

That is, the present device 100 provides an augmented reality image including the updated first user avatar to be shared, so that the first user does not expose his or her real appearance (while reducing personal privacy exposure) Through exposure of the first user avatar created to resemble the user, he or she can indirectly express himself, and through this, the first user's daily life and emotions can be actively shared with others without burden.

In addition, the screen control unit 220, when the real image 51 is displayed on the screen of the user terminal (first user terminal) for a preset time (for example, 3 seconds), in relation to the real image 51, SNS relational network By analyzing the uploaded information uploaded to the , it is possible to determine whether information related to a recommended visit place for recommending a visit to a place located in the vicinity of the real image 51 in relation to the real image 51 exists. Here, the upload information may include various types of information uploaded in relation to logs, postings, posts, etc. as described above, and may include posts (contents) related to blogs, postings, posts, and the like. In addition, the upload information may include message information transmitted and received in the user terminal (the first user terminal), but is not limited thereto.

In this case, if there is as a result of the determination (that is, if there is information related to a recommended place to visit that recommends visiting a place located in the vicinity of the real image 51), the screen controller 220 controls the manager avatar including information related to the recommended place to visit. can be generated and displayed on the real image 51 . That is, the screen controller 220 may display the first user avatar on the real image 51 . In this case, if it is determined that information related to a recommended visit place exists in relation to the real image 51 , the real image 51 ), in addition to displaying the first user avatar, an administrator avatar may be further displayed.

At this time, the screen control unit 220, when a user input (a first user's input) for the manager avatar displayed on the real image 51 displayed on the screen of the first user terminal 1 is made, the user terminal 1 ), information related to a recommended visit place included in the manager avatar may be displayed in a pop-up form, for example.

Here, when a plurality of place items are included in the recommended visit place related information, the screen controller 220 sets the display order of the plurality of place items in the recommended visit place related information displayed in a pop-up form, the plurality of user terminals 1 , 1', ...) may be updated in consideration of the level (level) of user action information for a plurality of place items. Hereinafter, the level may be expressed differently as a level or the like.

For example, when information for recommending four places is included in the recommended visiting place-related information, the plurality of place items includes a first place item recommending visiting a first place among the four places, and a visit of a second place. A second place item recommending , a third place item recommending visiting a third place, and a fourth place item recommending visiting a fourth place may be included.

In addition, a plurality of place items included in the recommended place-related information include, for example, places of famous tourist destinations, tourist course information of famous tourist destinations, landmarks (eg, museums, art galleries, etc.) places to visit in the city, places to eat, and restaurants. Information about various types of places (or courses), such as places, may be included.

In addition, the place item is information about the corresponding place, and may include, but is not limited to, name, address, route guide information, map information showing a location, usage fee information, usage time information, and the like.

Levels (levels) of user motion information considered when updating the display order of a plurality of place items may be divided into a plurality of levels including a first level, a second level, and a third level. In this case, in describing the level of user motion information below, the user terminal may refer to any one user terminal among the plurality of user terminals 1, 1', ....

The user action information corresponding to the first level may mean information about a user action of performing a selection input operation of selecting at least one place item from among a plurality of place items from the user terminal. The user action information corresponding to the second level may include any one place item in relation to any one place item among at least one place item selected by the user terminal (ie, at least one place item to which a selection input operation was made). It may mean information about a user action of performing a route guidance request input operation for requesting route guidance to a place corresponding to . The user motion information corresponding to the third level is a place where the user corresponds to any one place item, based on the route guidance information to a place corresponding to any one place item generated in response to a way guidance request input operation. It may mean information about a user operation of performing a place image acquisition operation of acquiring an image related to a place corresponding to any one place item as the user actually visits .

The screen control unit 220 may give different scores according to the level of user action information made with respect to the plurality of place items, and a plurality of place items in the recommended visit place related information displayed in a pop-up form based on the assigned scores. display ranking (ranking) can be updated.

For example, the screen controller 220 gives a first score to a place item in which user motion information corresponding to the first level is made, and gives a second score to a place item in which user motion information corresponding to the second level is made, , a third score may be given to a place item in which user motion information corresponding to the third level is made. In this case, the second score may be set higher than the first score, and the third score may be set higher than the second score.

At this time, when the third score is given, the screen control unit 220 includes location information (first location) of an image related to a place obtained based on the location image acquisition operation and location information ( second location), and if it is determined (verified) that the first location falls within a preset radius from the second location as a result of the comparison, it is determined that the user actually visited the place where the navigation request input operation was performed, and a third score is obtained can be given If, as a result of the comparison, the first location does not fall within a preset radius from the second location, it is determined that the user has not actually visited the place where the navigation request input operation is performed, and a third score may not be given.

For example, when the first user inputs to the manager avatar displayed on the screen of the first user terminal 1 , in one area of the screen of the first user terminal 1 , there are four place items as a plurality of place items. Information on recommended places to visit, including , can be displayed in a pop-up form.

In this case, it is assumed that, with respect to the first place item among the four place items, a selection input operation of selecting the first place item by a third user and a fourth user among a plurality of users has been performed in the past. In this case, as the level of user action information for the first place item made by the user terminal (third user terminal) of the third user is the first level, the first place item has a first score by the third user terminal can be given. Similarly, as the level of user action information for the first place item made by the user terminal of the fourth user (the fourth user terminal) is the first level, the first place item has a first score by the fourth user terminal can be given

If, with respect to the first place item, after the third user performs a selection input operation of selecting the first place item in the past, a navigation request input operation for requesting directions to a place corresponding to the first place item Let's say you've done In this case, as the level of user action information for the first place item made by the third user terminal is the second level, the first place item will be given a second score higher than the first score by the third user terminal. can

In addition, as an example, after the third user performs a route guidance request input operation for requesting route guidance to a place (ie, the first place) corresponding to the first place item, the third user provided in response to the route guidance request input operation Assume that you have actually visited the first place based on the route guidance information to the first place. Thereafter, in order to commemorate the first place that the third user actually visited, it is assumed that the image related to the first place is acquired through the image sensor of the third user terminal (that is, the third user acquires the image related to the first place) Let's say we have performed a place image acquisition operation). In this case, as the level of user action information for the first place item made by the third user terminal is the third level, the first place item will be given a third score higher than the second score by the third user terminal. can

As such, the screen control unit 220 gives a score according to the level of user action information by a plurality of users made by a plurality of users (or a plurality of user terminals) with respect to a plurality of place items, and gives a score accordingly It is possible to update the ranking (ranking) of a plurality of place items based on the obtained score. Thereafter, the screen controller 220 may display, on the screen of the user terminal (the first user terminal), information related to a recommended visit place including a plurality of place items in which the updated ranking (ranking) is reflected in a pop-up form.

That is, the screen controller 220 may update the display order of the plurality of place items in the recommended visit place related information in consideration of the level of user motion information by the plurality of users.

This device 100 performs a series of user actions (user action information) in which both online and offline actions are taken into consideration until the user's interest in a specific place (a list of specific places) is transferred to an actual action. By classifying a plurality of levels (levels) and performing rank update for a plurality of recommended visit places by giving different weights for each operation level, the reliability of the recommended place rank is secured and the reliability can be provided.

That is, in relation to a plurality of place items provided in a pop-up form, the device 100 gives (adds) a first score to a place item in which a selection input operation is performed among the plurality of place items, and a place where the selection input operation is performed Among the items, a second score higher than the first score is given to a place item in which a navigation request input operation, which is an action action of a user requesting a route guidance to visit a place corresponding to a specific place entry, is made, and the location from which the route guidance is requested Among the items, the user actually visits a place corresponding to the place item requested for directions and furthermore, through the user terminal, the user's action action to obtain an image at the place corresponding to the place item requested for way guidance is the place image obtaining action. By assigning a third score higher than the second score to the created place item, the ranking (display order, rank) of the plurality of place items may be updated in consideration of the level of user action information.

The present device 100 provides recommended visit place related information including a plurality of place items for recommending a visit to a specific place located in the vicinity of the shooting target area (or real image), among the plurality of place items. In order to preferentially display or recommend a more preferred place item, as an action (action) according to the user's interest in a specific place item, a series of steps from an online interest action to an actual offline interest action Action (i.e., the first action of selecting a place item → the second action of requesting directions for the selected place item → actually visiting the place for the place item for which the direction request is made, as well as the image of the place based on the action The third motion to be obtained) is divided by motion step (level, level) and different weights are given according to each motion level (in particular, the higher the motion level, the higher the score is given), so that the user motion information By assigning different weights to each, it is possible to update the rank of a plurality of recommended place items.

The apparatus 100 may prevent ranking (rank) manipulation for a plurality of place items and secure reliability of ranking by varying the weight of the score given according to the level of user motion information. In addition, the device 100 can provide reliability to the visit value of a place corresponding to the corresponding place item, and can attract visitors through this, thereby actively inducing promotion and activation of the corresponding area (place). In addition, as the device 100 provides route guidance information for a selected route guidance request place item among a plurality of place items, based on a route guidance request based on selection of a place item without inputting an address or name of the corresponding place It is possible to guide a route to a place corresponding to the corresponding place item, so that a user who is active online can be led to a visitor to an actual place offline.

This device 100 provides people with another space that can be shown to others, and can provide a part that is difficult to implement in reality or has limitations using augmented reality (AR) so that it can be implemented in virtual reality. . In addition, the device 100 may include elements of an existing light-weight SNS.

The conventionally known SNS uses a daily sharing method limited to a photo format and a short video. This has limitations in expressing information, and there is a problem with risks such as exposure of personal information.

On the other hand, the present device 100 introduces popular methods such as avatar decorating in the past to the recently used AR function, and proposes a more subdivided and new method of providing SNS. The device 100, for users who use the application provided through the device 100, displays their avatar (user avatar) on the screen when the user obtains an image of the place he or she has been to. It is possible to obtain an augmented reality image (photo, video, etc.) including an avatar. In addition, the device 100 adopts a recording method such that, when a user visits a specific place and acquires an augmented reality image for the visited place, the user's avatar is left in the corresponding place where the augmented reality image is acquired.

This device 100 may provide (help) users to more actively share their daily lives while reducing the fear of personal information exposure. In addition, the device 100 allows the user avatar to be created compared to the conventional SNS to express more diverse personalities.

The device 100 may provide a new type of SNS service providing technology that does not currently exist. The device 100 can provide an effect that can more fully satisfy the desire to show off, which is an essential reason for SNS. In addition, the device 100 can help the user to keep memories in a more enjoyable way.

In addition, since the SNS service providing technology by the device 100 is a concept that, when an augmented reality image is acquired at a place visited by the user, the user avatar is left in the corresponding place where the augmented reality image is acquired, so this is expected There may be a problem of the possibility of a more sensitive fight situation due to an unexpected situation (eg, harassment, provocation using gestures, etc.). Accordingly, the apparatus 100 performs the combination between the user avatar and the real image in consideration of the condition for not being able to combine the user avatar and/or determines whether to acquire the augmented reality image including the user avatar, as described above. It can eliminate the possibility of a conflict situation arising from an unexpected situation.

Conventional SNS technology, such as Facebook, provides various functions, but constantly causes problems in various advertisements and personal privacy issues. In addition, there are cases in which the conventional SNS technology cannot satisfy more needs of some people because there are clear limitations such as images and photos in expression methods.

Accordingly, the device 100 may be provided to the user as an application based on the concept of providing virtual reality with augmented reality.

The device 100 provides, for example, a personal profile management setting function, a personal timeline function, a photo and video upload function, a following/following function for a friend, a user avatar management function, and a decorative item of the user avatar (eg, clothing, Accessories, etc.) for sales/trading/production functions, AR photo (ie, augmented reality image) acquisition function, and ability to take pictures with a distant friend as an avatar (for example, a first user whose second user is another user other than the second user’s avatar) Function to acquire augmented reality image with user avatar), communication function with other users (communication function, comment function, empathy function), and random friend recommendation function tailored to the topic being uploaded, etc. It is not, and various functions may be included.

The device 100 may provide an SNS application using AR. The device 100 may provide an avatar set by the user in a picture taken by the user using the user terminal by applying AR to the SNS application so that the user can take the picture.

In addition, the device 100 may provide a user's avatar to be displayed on another user's terminal when another user visits by leaving a visit record at a place visited by the user based on the user's location, such as GPS. In addition, the device 100 may provide a picture to be taken with another user at a distance by using the user's avatar. Also, the device 100 may provide a personal profile management, timeline, photo and video upload, avatar management and decorating function, and communication function.

In addition, the device 100 may recognize an object, such as an object, an environment, etc. in an image when taking a photo or a video, and provide the recognized object, such as an object or an environment, to interact with the user's avatar (user avatar). That is, the apparatus 100 may change the posture, facial expression, etc. of the user avatar according to the recognized object.

In addition, the device 100 can variously change the operation (posture), facial expression, comments, etc. of the user avatar displayed on the screen in connection with the user's bio-signals, posting contents such as SNS blogs, weather, messages in the user terminal, etc. In addition, when taking photos, videos, etc., the user avatar is left in a location corresponding to the place (visited place) where the shooting takes place (the second storage unit stores the augmented reality image by linking and storing the user avatar, location can be recorded).

Also, the apparatus 100 may enable an image including an avatar, for example, a video call using the avatar, a game using the avatar, and the like.

In addition, the apparatus 100 may create a user avatar by selection/setting of the user when generating the user avatar. In addition, when the device 100 receives a user's photo (or video) when generating a user avatar, as an example, the user's characteristics (expression, skin tone, face type) in the received photo are analyzed through image analysis of the received photo. etc.) can be extracted. Thereafter, the apparatus 100 may acquire the user's body characteristic information in consideration of the extracted user's characteristics and, for example, information about the user's hobbies, age, gender, etc. previously inputted from the user, and the obtained body characteristics A user avatar in which information is reflected may be created.

Also, the apparatus 100 may generate and display a manager avatar other than the user avatar. The device 100 may provide information on famous tourist destination tourist courses and landmarks (museums, art galleries, etc.) worth visiting in the city center by using the manager avatar, as well as information on recommended places to visit. It is possible to create and provide a recommended route that can be checked.

In addition, the device 100 may use an avatar of a product (product) or service provider to be provided publicity to locate and record the avatar in a location near its own store, and store access information through the recorded avatar , product and service information, etc. may be provided.

Meanwhile, in the device 100 , the avatar manager 210 generates clothing recommendation information applied to the first user avatar when generating a user avatar (first user avatar) corresponding to the user (first user), and , a first user avatar on which clothing is worn corresponding to the generated clothing recommendation information may be generated and stored in the first storage unit 250 . When it is detected that the user terminal 1 is in a shooting mode to acquire an augmented reality image, the avatar manager 210 is configured to perform a first user avatar wearing clothes corresponding to the clothes recommendation information generated in response to the sensing of the shooting mode. may be generated and provided to the screen controller 220 so that the generated first user avatar is displayed on the real projection. A more detailed description thereof may be more easily understood with reference to FIG. 5 .

5 is a diagram schematically showing the configuration of the avatar manager 210 in the apparatus 100 for providing a social network service using augmented reality according to an embodiment of the present application.

Referring to FIG. 5 , the avatar management unit 210 includes the sensing unit 110 , the generating unit 120 , the management control unit 130 , the location determining unit 140 , the owned clothing management unit 150 , and the schedule determining unit 160 . , a history storage unit 170 and an owned clothing storage unit 180 may be included.

The sensing unit 110 may detect a point in time at which a sensing mode (photographing mode state) for the user terminal 1 to acquire an augmented reality image corresponding to the shooting target area is sensed. That is, the sensing unit 110 may detect, identify, and recognize the shooting mode detection time.

The generating unit 120 responds to the shooting mode detection time sensed by the sensing unit 110 , and captures the user's sensible characteristic information obtained through analysis of a plurality of worn clothing history data pre-stored in the history storage unit 170 . Considering the weather forecast information corresponding to the mode detection time, it is possible to generate clothing recommendation information corresponding to the photographing mode detection time.

Here, the worn clothing history data stored in the history storage unit 170 is a user input (which will be described later, for convenience of explanation) to an input request message (an input request message for input of sensory temperature information) to be described later. When a related user input) is made, the input value of the sensible temperature information is linked to at least one of the temperature measurement value of the user's surrounding environment, the weather forecast information at the time of input, and the attribute information of the worn clothes to form a set It may mean data that is configured and stored. A more detailed description thereof will be given later.

Although not shown in the drawing, the avatar manager 210 may include an analysis unit (not shown). The analysis unit (not shown) may analyze a plurality of worn clothing history data previously stored in the history storage unit 170 to provide user sensible characteristic information.

In this case, the user's sensible characteristic information is information obtained by analyzing correlation information (correlation, relevance) between at least one of a temperature measurement value of the user's surrounding environment, weather forecast information at the time of input, and attribute information of worn clothes and an input value of sensible temperature information. can mean Here, the sensible temperature information may refer to information about a degree of heat and/or a degree of cold that the user relatively senses (feels). Therefore, the user's sensible characteristic information refers to information obtained by analyzing the correlation between at least one of a temperature measurement value of the surrounding environment, weather forecast information, and attribute information of worn clothes, and an input value of sensible temperature information received by the user (correlation relationship). information) can be That is, the user's sensible characteristic information may mean correlation information related to the user's sensible temperature information.

The analysis unit (not shown) may use various correlation analysis algorithms previously known or developed in the future as an analysis algorithm when analyzing correlation information, but is not limited thereto.

As an example, the analysis unit (not shown) learns a plurality of pre-stored worn clothing history data as a deep learning model based on a plurality of pre-stored worn clothing history data in the history storage unit 170, thereby learning a deep learning model can be used to obtain user sensible characteristic information.

In this case, the deep learning model uses at least one of a temperature measurement value of the user's surrounding environment, weather forecast information at the time of input, and attribute information of the worn clothes as an input value from a plurality of worn clothes history data, and wears clothes history matching each of the input values. It may be a model trained to use the input value of the sensible temperature information in the data as an output value. By using such a pre-learned deep learning model, user sensible characteristic information (correlation information between each data stored as wearable clothing history data) may be obtained.

Here, the deep learning model may mean an artificial intelligence (AI) algorithm model, a machine learning (machine learning) model, a neural network model (an artificial neural network model), a neurofuzzy model, and the like. In addition, the deep learning model is exemplarily a convolutional neural network (CNN, convolutional neural network), a recurrent neural network (RNN), a deep neural network, etc. that have been previously known or developed in the future. Various neural network models can be applied.

The weather forecast information considered herein may be obtained, for example, from a meteorological agency server (not shown) connected to the apparatus 100 and the network 2 through the network 2 . The Korea Meteorological Administration server (not shown) may provide various types of information related to weather, including weather forecast information.

The generating unit 120 corresponds to the user's sensible characteristic information (correlation information related to the user's sensible temperature information) provided by the analysis of the analysis unit (not shown) and the shooting mode detection time obtained from the Meteorological Administration server (not shown) In consideration of the weather forecast information, it is possible to generate clothing recommendation information corresponding to the time of sensing the shooting mode.

The generator 120 may display the generated clothing recommendation information as customized clothing recommendation information customized to the user, for example, on the screen of the user terminal. Also, the generation unit 120 puts the generated clothing recommendation information on the user avatar (first user avatar) generated by the avatar management unit 210 , so that the first clothing corresponding to the generated clothing recommendation information is worn. A user avatar (ie, a first user avatar wearing clothes) may be generated and stored in the first storage unit 250 . In addition, in response to the sensing time of the shooting mode, the screen controller 220 controls the first user avatar (that is, the first user wearing clothes) on which clothes are worn corresponding to the clothes recommendation information generated by the generator 120 . avatar) to be displayed on the screen, the generator 120 may transmit the first user avatar wearing clothes to the screen controller 220 . Thereafter, the screen controller 220 may overlap and display the first user avatar wearing clothes generated by the avatar manager 210 on the real image displayed on the screen of the user terminal 1 .

The clothing recommendation information generated by the generator 120 may refer to information for recommending clothing (dressing, coordinating) to be worn by the user avatar displayed on the screen of the user terminal 1 at the time of sensing the shooting mode.

In an example of the present disclosure, the clothing recommendation information generated by the generator 120 is exemplified as information for recommending clothing worn (applied) to the user avatar, but is not limited thereto. As another example, the clothing recommendation information generated by the generator 120 refers to information that recommends clothing (suitable for clothes, coordination) suitable for the user to actually wear at the time (date) corresponding to the detection time of the shooting mode. can do. That is, the clothing recommendation information may refer to customized clothing recommendation information for which the user actually recommends wearing.

The management control unit 130 is configured to display the screen of the user terminal 1 at every preset time period when some owned clothes are selected as wearing clothes among the information on the clothes actually owned by the user stored in the owned clothes storage unit 180 to be described later. An input request message for input of sensible temperature information may be displayed in a region of the . In particular, the management control unit 130 allows the user to select some owned clothing from among the information about the owned clothing stored in the owned clothing storage unit 180 as wearing clothing, and actually wears real clothing corresponding to the selected worn clothing, and then performs an external activity. In the case of performing the external activity, an input request message may be displayed on one area of the screen at each preset time period during the execution of the external activity.

Here, the term "outside activity" may mean an act of walking around wearing clothes, such as going out. In addition, the preset time period may be set to, for example, 2 hours or 3 hours, but is not limited thereto. In an example of the present application, it is illustrated that the preset time period is set in units of time, but this is only an example for helping understanding of the present application and is not limited thereto, and is set in various units such as units of seconds, units of days, etc. can be

The management control unit 130 may display an input request message for input of the sensible temperature information on the screen. Here, the sensible temperature information may refer to information about the degree of heat and/or the degree of cold that the user relatively senses (feels) in comparison with a temperature measurement value of the user's surrounding environment.

Even if the weather temperature is the same, the relative sensible temperature felt (feeled) by each user at the corresponding temperature is different. For example, let's say the current weather temperature is 22 degrees. At this time, if the first user has a constitution that is prone to cold and is wearing a thick long-sleeved top/long bottom, he/she may feel warm as a sensible temperature (appropriate temperature). As another example, when the second user has a constitution that is prone to cold and is wearing a short-sleeved top (cotton tee)/short-sleeved bottoms, he or she may feel cold as a sensible temperature. As another example, if the third user has a constitution that burns a lot of heat and is wearing a short-sleeved top/short-sleeved bottom, he or she may feel hot as a sensible temperature.

As such, the sensible temperature felt by each user may be different depending on the user's original constitution or surrounding environment/situation. However, since the conventional clothing recommendation technology generally provides average clothing (dressing, coordination) recommendation information for a specific temperature, the average clothing recommendation information may be treated as unsatisfactory information for some users.

Accordingly, the device 100 may display an input request message on the screen at every preset time period to obtain the sensible temperature information in order to provide customized clothing recommendation information optimized to the user based on the sensible temperature information felt by the user. .

In addition, the device 100 applies clothing corresponding to clothing recommendation information (custom clothing recommendation information) generated based on the sensible temperature information felt by the user to the first user avatar and displays it on the screen. As the included first user avatar, the application of the first user avatar in which the user's current feeling to the user's current state and sensible temperature information is applied may be provided. The apparatus 100 may provide the first user avatar with an avatar having a feeling very similar to that of an actual user.

The input request message may mean a message for requesting a user input (ie, a user input related to the first query) for a query asking whether the user is currently cold or hot.

The input request message may be, for example, a message including options such as 'cold', 'hot', and 'suitable' along with a query of 'what are the clothes you are wearing at the current temperature?' In an example of the present application, three options for input of sensible temperature information have been exemplified as an example (cold, hot, suitable), but is not limited thereto.

For example, the options for inputting the sensible temperature information are divided into 10 steps, where a lower level may mean stronger coldness, and a higher level may mean stronger heat. That is, level 1 may mean very cold, level 4 a little cold, level 5 moderate, level 5 a little hot, level 10 very cold, etc.

In response to the display of the input request message on the screen, the user may input sensible temperature information that the user feels at a current time point (ie, a time point at which the input request message is displayed). That is, the user may select any one of a plurality of options related to the sensible temperature information in response to the message display.

Thereafter, when a user input (a first query-related user input) for the input request message is made, the management control unit 130 responds to the first query-related user input, and the user corresponding to the time point at which the first query-related user input is made. At least one of the temperature measurement value of the surrounding environment, the weather forecast information at the input time corresponding to the time when the user input related to the first query is made, and attribute information of the worn clothes corresponding to the time when the user input related to the first query is made, in the input request message In connection with the input value of the corresponding sensible temperature information, it may be controlled to be stored in the history storage unit 170 as worn clothing history data.

Here, the time point at which the first query-related user input is made may mean a time point (current time point) at which the sensible temperature information is input by the user in response to the input request message displayed on the screen.

The temperature measurement value of the user's surrounding environment may mean a value obtained by measuring the temperature of the surrounding (neighboring environment) where the user is located at the point in time when the user input related to the first query is made. The temperature measurement value of the user's surrounding environment is, for example, a temperature measuring sensor (not shown) provided in the device 100, a temperature measuring sensor (not shown) provided in the user terminal 1 carried by the user, and the device It may be obtained using at least one of the temperature measuring sensors that are connected through the network 100 and the network 2 and are provided (installed) in the space where the user is located.

The input time weather forecast information may refer to weather forecast information corresponding to the date at which the first query related user input is made.

The attribute information of the worn clothing is attribute information on the clothing (wearing clothing) that the user is actually wearing at the time when the user input related to the first query is made, and includes the type (type), material (material), thickness, and size ( dimension, length), color (color), and the like, but is not limited thereto. Here, the type of clothing to be worn may mean information for classifying whether it is a short sleeve, a long sleeve, a skirt, an outer, and the like.

The input value of the sensible temperature information may mean a value of any one selected by the user from among a plurality of options related to the sensible temperature information.

As such, whenever an input request message is displayed on the screen at a preset time period, the management control unit 130 responds to a user input (a user input related to the first query) corresponding to the temperature measurement value of the user's surrounding environment and weather forecast at the input time At least one of the information and the attribute information of the worn clothing may be linked with the input value of the sensible temperature information to generate and store the worn clothing history data in the history storage unit 170 . In this case, if schedule information (schedule information including acquaintance information) to be described later exists in the worn clothing history data, the schedule information may be linked and stored.

The history storage unit 170 may accumulate and store worn clothing history data generated in response to a user input related to a first query at each preset time period. Thereafter, the analysis unit (not shown) analyzes a plurality of worn clothing history data (a plurality of accumulated worn clothing history data) stored in the history storage unit 170 (in particular, performs correlation analysis) to obtain user experience characteristic information. can be obtained Thereafter, the generator 120 may generate the clothing recommendation information corresponding to the sensing time of the shooting mode in consideration of the acquired user sensible characteristic information and weather forecast information.

As the user sensible characteristic information, which is the above-described correlation information, is obtained through analysis by an analysis unit (not shown), for example, the generation unit 120 obtains from the Korea Meteorological Administration server (not shown) when a shooting mode detection time is detected. If there is weather forecast information corresponding to the sensing time of the shooting mode, information that 'the user will feel some degree of cold (or heat) on the date corresponding to the sensing time of the shooting mode' is provided based on the user's sensible characteristic information. Predicted sensible temperature prediction information may be acquired, and clothing recommendation information corresponding to the sensing time of the shooting mode may be generated based on the acquired sensible temperature prediction information.

As another example, it is assumed that, when the sensing time of the shooting mode is sensed, weather forecast information corresponding to the sensing time of the shooting mode obtained from the Meteorological Administration server (not shown) and selected clothing information exist. Here, the selected clothing information may refer to information on clothing that the user desires to wear on a date corresponding to the sensing time of the shooting mode, among the owned clothing stored in the owned clothing storage unit 180 . In this case, if the generation unit 120 actually wears and wears clothes corresponding to the selected clothes information on the date corresponding to the time of sensing the shooting mode, the user will feel very hot (or a lot of It will be cold), and predicted information (ie, sensible temperature prediction information) may be obtained, and clothing recommendation information corresponding to the sensing time of the shooting mode may be generated based on the acquired sensible temperature prediction information.

The location determination unit 140 may determine whether the user is located outdoors or indoors based on location information of the user (the first user) obtained through a location sensor (not shown).

Here, the position measuring sensor (not shown) may mean, for example, at least one of a position measuring sensor provided by the device 100 and a position measuring sensor provided in the user terminal 1 carried by the user. The position measuring sensor (not shown) may be referred to as a GPS sensor or the like.

The management control unit 130 stores the worn clothing history data in the history storage unit 170 in response to a user input for displaying an input request message according to the position determination result by the position determination unit 140 , and forecasts the weather at the input time. Information can be optionally included and stored.

For example, when it is determined that the user is located outdoors as a result of the location determination, the management controller 130 may include weather forecast information at the time of input when storing the worn clothing history data in the history storage unit 170 . On the other hand, when it is determined that the user is located indoors as a result of the location determination, the management control unit 130 may not include the weather forecast information at the time of input when storing the worn clothing history data in the history storage unit 170 .

When the user is located outdoors, it can be said that the user is directly affected by the weather forecast information because the user is exposed to the outdoors. On the other hand, when a user is located indoors, a limited space is formed due to a structure indoors, and as an air conditioner or heater is operated in the space, it is not significantly affected by weather forecast information, but rather measures the temperature of the user's surrounding environment. It can be said that the state is greatly affected by the value.

That is, it can be said that the weather forecast information responds more sensitively to (influenced by) the sensible temperature information, which is the temperature the user feels when the user is outdoors than when the user is indoors.

Accordingly, when the management control unit 130 stores the worn clothing history data in the history storage unit 170 in response to a user input to the input request message, the input timing depends on whether the user is located outdoors or indoors. Weather forecast information can be optionally included and stored. That is, when it is determined that the user is located outdoors, the management control unit 130 stores the worn clothing history data by including the input time forecast information together, and when the user is located indoors, excludes the input time forecast information and Wearing clothing history data may be stored. As such, the present apparatus 100 can obtain more reliable and accurate user's sensible characteristic information by analyzing and acquiring the user's sensible temperature information in consideration of the user's sensible temperature information. It is possible to provide customized clothing recommendation information.

The preset time period during which the input request message is displayed on the screen by the management control unit 130 is based on the position determination result determined by the position determination unit 140 when the user is located indoors compared to when the user is outdoors. It can be set shorter when located.

That is, the preset time period for displaying the input request message may be set to a first period when the user is located indoors, and may be set to a second period when the user is located outdoors. In this case, the first period may be set shorter (set to a smaller value) than the second period. For example, the first period may be set to 1 hour, and the second period may be set to 2 hours longer than the first period.

According to this, when it is determined that the user is located outdoors, the management control unit 130 displays an input request message on the screen in units of one hour, and when it is determined that the user is located indoors, the input request message is sent to 2 It can be displayed on the screen in units of time periods.

In the present device 100, when the user is outdoors, the possibility of being more active (possibility of being in motion) (for example, the likelihood of being in motion, such as walking or running) is higher when the user is outdoors than when the user is indoors. It can be determined that the change in the sensible temperature is more likely to vary more fluidly when outdoors than when indoors. Accordingly, the device 100 varies the preset time period for displaying the input request message according to whether the user's location analyzed by the analysis unit (not shown) is outdoors or outdoors, thereby reducing the sensible temperature experienced by the user. By reflecting changes in real time, it is possible to provide more optimized customized clothing recommendation information.

The generator 120 considers at least one of the user's personal hospital visit history information, the user's biometric information, and the user's constitution information, in addition to generating the clothing recommendation information in consideration of the user's sensible characteristic information and the weather forecast information. Thus, it is possible to generate clothing recommendation information corresponding to the detection time of the shooting mode.

Here, the hospital visit history information may include, for example, information related to whether or not a cold occurs, information related to the invention of a headache, and the like, but is not limited thereto. Hospital visit history information is, for example, from an EMR server (not shown) and/or a hospital server (not shown) that records an Electronic Medical Record (EMR) connected through the device 100 and the network 2 It can be obtained through analysis of the obtained electronic medical record.

The user's biometric information may refer to various types of information related to the user's characteristics including, for example, the user's blood pressure, body temperature, and ECG signal. The user's biometric information may be obtained from, for example, a wearable device (not shown) that is connected to the present apparatus 100 and the network 2 through the network 2 and is worn on a part of the user's body. Here, the wearable device (not shown) may include, but is not limited to, a smart watch, and any device that is provided to be worn on at least a part of the user's body, such as a watch, glasses, and a headset, is applicable. do.

The constitutional information of the user may include, for example, information on preferred food, allergy information, and the like. Such constitutional information of the user may be obtained by, for example, the user directly typing through the user terminal 1, or may be obtained from the electronic medical record recorded in the above-described EMR server (not shown).

Exemplarily, when the management control unit 130 wants to store the worn clothing history data in the history storage unit 170 , the EMR server (not shown) and/or the hospital server ( By analyzing the electronic medical record obtained from (not shown), it is possible to check whether the user's personal hospital visit history information exists. In this case, as a result of the confirmation, it is assumed that there is information related to whether or not a cold has occurred in relation to the user's personal hospital visit history information, and as a result of analyzing this information, it is confirmed that the user is currently ill with a cold. In this case, when storing the worn clothing history data, the management control unit 130 receives, for example, at least one of a temperature measurement value of the user's surrounding environment, weather forecast information at the time of input, and attribute information of the worn clothing and an input value of the sensible temperature information. In addition to the information linked to each other, information related to the onset of a cold indicating that 'the user is currently catching a cold' is linked together as the wearing clothing history data (for example, the first wearing clothing history data), the history storage unit 170 can be stored in

According to this, for example, the worn clothing history data (eg, the first worn clothing history data) stored in the history storage unit 170 is 'according to the temperature measurement value of the user's surrounding environment where the user is located, the temperature around the current user is 19 According to the attribute information of the currently worn clothing worn by the user, the user is wearing a cotton long-sleeved top and cotton shorts with a thickness of 3 mm. The weather forecast information for the date is 20 degrees, and according to the user's personal hospital visit history information, it is information indicating that the user is currently in a cold state, and the input value of the sensible temperature information. The temperature information may be, for example, information in which information corresponding to 'slightly hot corresponding to step 5' is linked to each other.

That is, in the history storage unit 170 , for example, when the user catches a cold, what is the temperature measurement value of the surrounding environment at that time, what is the weather forecast information at that time, and what is the attribute information of the clothes worn at that time? When , information indicating what kind of information about the degree of heat and/or degree of cold that the user has relatively felt (felt) may be stored as the first worn clothing history data.

The analysis unit (not shown) analyzes a plurality of worn clothing history data, such as the first worn clothing history data stored in the history storage unit 170 , so that different types of data stored in the history storage unit 170 (for example, User sensible characteristic information indicating the correlation between five types of data, including temperature measurement values of the user's surrounding environment, weather forecast information at the time of input, attribute information of worn clothing, input values of sensible temperature information, and information on whether or not a cold occurs) can be obtained.

Thereafter, when the detection time of the shooting mode is detected, the generation unit 120 determines that 'the user is currently catching a cold' as an example of the analysis result of the electronic medical record, and weather forecast information corresponding to the time information is obtained. In this case, it is possible to obtain sensible temperature prediction information that 'the user will feel a little cold on the date corresponding to the time of sensing the shooting mode' based on the user's sensible characteristic information obtained by the analysis unit (not shown). have. Thereafter, the generation unit 120 determines that it is most suitable to wear when the user feels a little cold among the plurality of owned clothing pre-stored in the collectible clothing storage unit 180 based on the obtained sensible temperature prediction information. At least one piece of clothing can be selected. Thereafter, the generator 120 may generate clothing recommendation information including the selected at least one piece of clothing, and provide the generated clothing recommendation information to the management controller 130 so that the generated clothing recommendation information is displayed on the screen as customized clothing recommendation information. In addition, the generation unit 120 generates a first user avatar wearing clothes corresponding to the generated clothing recommendation information, stores it in the first storage unit 250, and the user terminal 1 to obtain an augmented reality image. ) can be overlapped and displayed on the real image displayed on the screen.

In the above-described example, when generating the clothing recommendation information by further considering at least one of the user's personal hospital visit history information, the user's biometric information, and the user's constitution information, the generating unit 120 exemplarily includes the user's personal information. A case in which clothing recommendation information is generated in consideration of hospital visit history information has been specifically exemplified. This is only an example to help the understanding of the present application, and is not limited thereto, and the process of generating clothing recommendation information in consideration of the user's biometric information and the user's constitution information considers the user's personal hospital visit history information Thus, in comparison with the process of generating the clothing recommendation information, the same or similar method may be used. Therefore, even if omitted below, the description of the process of generating the clothing recommendation information in consideration of the user's personal hospital visit history information differs only in the consideration subject, but the process of generating the clothing recommendation information in consideration of the user's biometric information and the user's constitution The same may be applied to a description of a process of generating clothing recommendation information in consideration of the information.

The owned clothing management unit 150 may obtain information about the owned clothing owned by the user, store it in the owned clothing storage unit 180 , and manage it.

Here, the owned clothes may mean clothes that the user actually owns (eg, clothes, pants, skirts, dresses, suits, etc.). Here, the information on the owned clothing may include at least one of image information on the corresponding clothing (owned clothing), purchase date information of the owned clothing, and attribute information of the owned clothing, but is not limited thereto. can

Here, the information on the owned clothing may be obtained by, for example, a user directly typing through the user terminal 1 . As another example, the information on the owned clothes may be obtained by the user photographing the owned clothes through an image sensor (not shown) of the user terminal 1 . As another example, when a user acquires image information on owned clothing through an image sensor (not shown) of the user terminal 1 , the information on owned clothing is obtained through image analysis on the acquired image information. It can be obtained by searching the Internet (or online shopping mall) for clothes (matching clothes) matching the owned clothes in the image information, and crawling (collecting) recorded information about the searched matching clothes. However, these examples are merely examples for helping understanding of the present disclosure, and the present disclosure is not limited thereto, and information on owned clothing may be obtained in various ways.

In other words, as an example, information on owned clothing is automatically entered by a user directly typing or when a user inputs part number information about owned clothing, matching information is searched for by linking with an online shopping mall or clothing company DB. can be input by extracting detailed information from the corresponding online shopping mall site. The information on the owned clothing may include image information of the owned clothing, manufacturing year information, purchase date information, color, color, size, and the like information.

The owned clothing management unit 150 satisfies a preset wear condition among the owned clothing (owned clothing) stored in the owned clothing storage unit 180 based on the analysis of a plurality of worn clothing history data previously stored in the history storage unit 170 . worn-out clothing can be identified.

Here, the preset wear-out condition may be, for example, a condition in which the number of times the number of times the user selects the clothes to be worn among the owned clothes(s) stored in the owned clothes storage unit 180 is equal to or greater than the preset number of times. The preset number of times may be set, for example, 80 times, 100 times, etc., and is not limited thereto and may be variously set.

As another example, the preset wear-out condition may be a condition in which a preset period has elapsed from a date corresponding to information on the purchase date of the owned clothing among the owned clothing(s) stored in the owned clothing storage unit 180 . Here, the information on the purchase date of the owned clothing may be identified from information about the owned clothing stored in the owned clothing storage unit 180 . In addition, the preset period may be, for example, three years, and is not limited thereto and may be variously set.

According to this, the owned clothing management unit 150 sets the number of wearing of the owned clothing stored in the owned clothing storage unit 180, for example, 80 times or more, or 3 years from the date of purchase, the predetermined wear of the owned clothing. It can be identified as a worn-out garment that meets the conditions.

The generation unit 120 generates a replacement clothing recommendation list for replacing the identified worn clothing in response to the identification of the worn clothing by the owned clothing management unit 150 and displays the generated replacement clothing recommendation list on the screen. It may be provided to the management control unit 130 .

That is, when the owned clothing management unit 150 identifies at least one piece of information on the owned clothing stored in the owned clothing storage unit 180 as worn clothing, the generation unit 120 responds to the identification of the worn clothing, A recommendation list (ie, a replacement clothing recommendation list) for recommending replacement clothing for replacing worn clothing with other clothing may be generated. Thereafter, the generation unit 120 may provide the generated replacement clothing recommendation list to the management control unit 130 to display it on the screen, and the management control unit 130 displays the provided replacement clothing recommendation list on one area of the screen. can do.

In other words, when worn clothing is identified by the owned clothing management unit 150 , the generation unit 120 may generate and provide a replacement clothing recommendation list for recommending replacement clothing that can be replaced in relation to the identified worn clothing. .

In this case, the replacement clothing included in the replacement clothing recommendation list may include at least one piece of clothing sold (traded) on the online shopping mall. In particular, the replacement clothing included in the replacement clothing recommendation list may include the same (same) replacement clothing, which is the same clothing as the worn-out clothing, and similar replacement clothing (or dissimilar replacement clothing), which is a different type of clothing with similar performance to the worn-out clothing. have.

In particular, similar replacement clothing may include clothing with better reviews (evaluation), lower price, or higher quality compared to worn out clothing determined to be replaced.

According to this, the apparatus 100 determines that replacement of the worn-out clothing is necessary for worn-out clothing that meets a preset wear-out condition among the owned clothing owned by the user, and provides the same or similar performance to that of the worn-out clothing. By providing a replacement clothing recommendation list including other clothing owned by the user, in determining clothing to replace worn clothing, the user can easily compare various clothing and select more rationally.

That is, when worn clothing is identified, the apparatus 100 determines that replacement (replacement) is necessary for the identified worn clothing, and may provide a replacement clothing recommendation list for replacement of the worn clothing. In this case, the worn clothing and It is possible to provide (recommend) not only the same clothing but also clothing of the same type but with good performance and good performance, included in the replacement clothing recommendation list as replacement clothing. This device 100 allows the user to rationally compare/select products when replacing (replacement) worn out clothing, and can support replacement (replacement) and management more easily and efficiently.

The schedule determination unit 160 determines whether pre-registered schedule information related to a meeting with an acquaintance (ie, schedule information related to a meeting with an acquaintance) exists on a date corresponding to the detection time of the shooting mode detected by the sensing unit 110 . can do.

Such schedule information may refer to, for example, information recorded in a schedule management menu (eg, a calendar menu) provided through the device 100 . For example, when the user promises to meet the first acquaintance among a plurality of acquaintances the next day, the user enters the schedule management menu provided through the device 100 and enters '3 pm with the first acquaintance on the date corresponding to the day after day. You can save (record) schedule information such as 'to meet at

In this case, it may be determined that pre-registration schedule information related to the meeting with the first acquaintance exists on the date corresponding to the time of sensing the shooting mode. As a result of determination by the schedule determining unit 160 , when there is pre-registration schedule information, the generating unit 120 determines whether previously registered acquaintances ( For example, it is possible to calculate acquaintance statistical information on the collection clothing worn when the first acquaintance) met, and regenerate clothing recommendation information based on the calculated acquaintance statistical information.

According to this, the analysis unit (not shown) can acquire the above-described user's sensible characteristic information through correlation analysis based on a plurality of pre-stored worn clothing history data in the history storage unit 170 , and also obtain the previously stored Statistical information may be generated by performing statistical analysis based on a plurality of worn clothing history data.

When storing the worn clothing history data in the history storage 170 , for example, when there is schedule information (particularly, schedule information including acquaintance information) corresponding to the time when the worn clothing history data is stored, the history storage unit At 170 , schedule information (particularly, schedule information including acquaintance information) may be linked and stored in the worn clothing history data.

That is, in the above-described example, at least one of the temperature measurement value of the user's surrounding environment, the weather forecast information at the time of input, and the attribute information of the worn clothing and the input value of the sensible temperature information are linked to each other to form a single data set, and the worn clothing history Although it has been exemplified as storing data in the history storage unit 170 , the present invention is not limited thereto, and when certain information including acquaintance information exists, worn clothing history data in which the predetermined information is further linked is stored in the history storage unit 170 . can be saved

Through this, the analysis unit (not shown) may generate statistical information through statistical analysis of a plurality of worn clothing history data stored in the history storage unit 170 .

Thereafter, when it is determined that, for example, pre-registered schedule information related to the first acquaintance exists on a specific date corresponding to the sensing time of the shooting mode, the generating unit 120 analyzes (statistical analysis) of the analyzing unit (not shown). Based on the statistical information generated through , it is possible to calculate acquaintance statistical information about the clothes owned by the wearer that was worn when meeting the first acquaintance in the past, and regenerate clothing recommendation information based on the calculated acquaintance statistical information.

In this case, the clothing recommendation information generated by the generator 120 in consideration of the user's sensible characteristic information and the weather forecast information without considering the schedule information (or statistical information) may be referred to as first clothing recommendation information. On the other hand, the clothing recommendation information generated in consideration of the schedule information (or statistical information) is different from the first clothing recommendation information, and may be referred to as second clothing recommendation information. That is, the generator 120 may regenerate the clothing recommendation information as the second clothing recommendation information instead of the first clothing recommendation information in consideration of the schedule information (or statistical information).

Specifically, the clothing recommendation information generated in consideration of the user's sensible characteristic information and the weather forecast information may be the first clothing recommendation information. Based on the calculated acquaintance statistical information, the generation unit 120 is configured to wear the owned clothing corresponding to the previously generated first clothing recommendation information (that is, to select the collected clothing corresponding to the first clothing recommendation information for the first time in the past). If it is determined that the number of times worn when meeting an acquaintance) exceeds the preset number of wearing, the previously generated clothing recommendation information (first clothing recommendation information) is regenerated into second clothing recommendation information different from the first clothing recommendation information. can Thereafter, the management controller 130 may display the regenerated second clothing recommendation information as customized clothing recommendation information on the screen.

Here, the preset number of wearing may be exemplarily set to 5, but is not limited thereto, and the number of times may be variously set.

According to this, the generation unit 120 generates the clothing that the collectible clothing corresponding to the first clothing recommendation information has worn several times (for example, 5 times or more, which is a preset number of times) when meeting a specific acquaintance (first acquaintance) in the past. In the case of , second clothing recommendation information including clothes different from the collected clothes corresponding to the first clothes recommendation information may be generated.

In another example, regardless of schedule information (statistical information of acquaintances), the generation unit 120 based on the statistical information obtained by the analysis of the analysis unit (not shown), for example, the collection corresponding to the first clothing recommendation information. If it is determined that the number of times of wearing clothes is greater than or equal to the preset number of times during a preset period, the clothes recommendation information may be regenerated as the second clothes recommendation information instead of the first clothes recommendation information. Here, the preset section period may be, for example, a period before one week or two weeks before the time when the clothing recommendation information is generated, and the like, but is not limited thereto. In addition, the preset number of times may be, for example, three times, but is not limited thereto.

Accordingly, when the collectible clothing corresponding to the first clothing recommendation information is clothing that has been recently worn multiple times (eg, three or more times a preset number of times) in a week, for example, the generation unit 120 recommends the first clothing. Second clothing recommendation information including clothing different from the collected clothing corresponding to the information may be generated.

In this case, the clothing recommendation information generated by the generator 120 (that is, clothing recommendation information including the first clothing recommendation information and the second clothing recommendation information) is stored in the owned clothing storage unit 180 of the owned clothing (owned). It may be information generated based on clothes). That is, the generator 120 may generate the clothing recommendation information based on the clothing (owned clothing) owned by the user.

In addition, as an example, the management control unit 130 includes, as a result of determination by the schedule determination unit 160 , pre-registered schedule information related to a meeting with an acquaintance (eg, the first acquaintance) with respect to a date corresponding to the time of detection of the shooting mode. If it is determined to do so, acquaintance information (first acquaintance information) about the acquaintance (eg, first acquaintance) included in the pre-registration schedule information is identified, and the first acquaintance information (first acquaintance information) is identified. of contact information can be extracted. Here, the acquaintance information may include the acquaintance's name, contact information, address, etc., but is not limited thereto and may include various information. Then, the management control unit 130, based on the extracted contact information of the first acquaintance, the user and the acquaintance (the first acquaintance) with each other, meeting related schedule information (ie, pre-registration related to the meeting with the acquaintance) schedule information) may be provided.

In this case, the management control unit 130 may provide the schedule notification message before a preset time from the date corresponding to the pre-registered schedule information (the date corresponding to the detection time of the shooting mode). Here, the preset time may exemplarily be 24 hours before, but is not limited thereto. In addition, the management control unit 130 may provide the schedule notification message to the user terminal 1 of the user who has stored the previously registered schedule information and to the terminal of the acquaintance (the first acquaintance) related to the previously registered schedule information.

That is, in providing the schedule notification message, the management control unit 130 does not provide schedule information related to a meeting with an acquaintance only to the user terminal 1 of the previously registered user, but the user terminal 1 and the pre-registered schedule. A schedule notification message may be provided to each terminal of the acquaintance corresponding to the contact information of the acquaintance included in the information.

According to this, the device 100 provides a schedule notification (schedule notification) for a meeting appointment between the user and the acquaintance (the first acquaintance), as well as the user who has registered/stored the schedule in his/her terminal. , it can be made known together (simultaneously) to all acquaintances (ie, first acquaintances) related to the corresponding schedule. In this device 100, even if the first acquaintance does not separately record or memorize a schedule related to a meeting (meeting appointment) with the user, the user has a pre-registered schedule registered in his/her terminal (user terminal). The information allows the first acquaintance to receive a notification of the schedule for the meeting (meeting appointment), so that not only the user (the user) but also the other party (eg, the first acquaintance) who has decided to make the meeting appointment together By reminding them not to forget the schedule, they can help make the meeting go smoothly.

That is, the management control unit 130 may provide a schedule notification message to each of the user terminal 1 and the terminal of the acquaintance. In this case, the device 100 provides, for example, a schedule notification message to the user terminal 1 (displaying a schedule notification message on the screen of the user terminal), and then confirms a message indicating that the provided schedule notification message is checked in response The signal may be received through a user input (for example, a touch input to the screen, etc.).

In this case, the management control unit 130 provides a schedule notification message to the user terminal 1, if the message confirmation signal is not received within a preset time (for example, 3 hours) (that is, the user with respect to the provided schedule notification message) is not inputted), the user may recognize that the provided schedule notification message has not been checked, and the notification provision type for providing the schedule notification message may be changed.

Here, changing the notification providing type means, for example, when the notification providing type of the schedule notification message is the current vibration notification, changing the vibration strength of the schedule notification message from the first vibration strength to the second vibration strength stronger than that can mean that According to this, the management control unit 130 manages when a message confirmation signal is not received within a preset time (for example, 3 hours) after primarily providing the schedule notification message to the user terminal 1 at the first vibration intensity. The controller 130 may change the vibration strength from the first vibration strength to the second vibration strength, and then provide (transmit) a schedule notification message having the changed second vibration strength to the user terminal 1 again.

As another example, changing the notification providing type may mean changing the notification providing type of the schedule notification message to a different type when, for example, the notification providing type of the schedule notification message is a current vibration notification. Here, the notification providing type may include at least one of a vibration notification, a sound notification, and a phone notification. According to this, if the management control unit 130 does not receive a message confirmation signal within a preset time (for example, 3 hours) after providing the schedule notification message to the user terminal 1 as a vibration notification primarily, the management control unit ( 130) may change the notification providing type from the vibration notification to the sound notification, and then provide a schedule notification message having the changed sound notification to the user terminal 1 again. According to the provision of the schedule notification message having such a sound notification, in response thereto, the user terminal 1 may output a sound notification such as 'a schedule notification message exists' through the speaker unit (not shown).

If the management control unit 130 does not receive a message confirmation signal within a preset time (for example, 3 hours) after providing the schedule notification message to the user terminal 1 as a sound notification, the management control unit 130 ) may change the notification providing type from a sound notification to a phone notification, and then provide a schedule notification message having the changed phone notification to the user terminal 1 again. According to the provision of the schedule notification message having such a call notification, an automatic answering phone connection such as an ARS is made to the user terminal 1 in response thereto, and the schedule notification message can be provided to the user through the ARS.

As such, the device 100 provides a changed notification by changing the notification providing type if a response (message confirmation signal) is not received within a preset time (for example, 3 hours) after providing the schedule notification message for the first time By providing the rescheduling reminder message in a tangible way, it can help the user to more accurately recognize (clearly recognize) the confirmation of the schedule reminder message.

In addition, in generating the clothing recommendation information, the generator 120 determines the temperature difference between night and day from the weather forecast information corresponding to the time of sensing the shooting mode obtained from the Meteorological Agency server (not shown), for example, and determines the temperature difference Depending on the level (degree) of the information, different clothing recommendation information may be generated. Here, night and day may mean a commonly defined time zone, but is not limited thereto.

Illustratively, the generating unit 120 obtains weather forecast information corresponding to the sensing time of the shooting mode from the Meteorological Administration server (not shown), and determines the temperature difference information between night and day from the obtained weather forecast information corresponding to tomorrow. can In this case, the generator 120 generates first clothing recommendation information when the determined temperature difference information falls within the first range, and generates second clothing recommendation information when the temperature difference information falls within the second range, and the temperature difference information If it falls within the 3 range, third clothing recommendation information may be generated.

Here, the first range means, for example, a range in which the temperature difference between night and day is 0 degrees or more and less than 5 degrees, and the second range means a range in which the temperature difference between night and day is 5 degrees or more and less than 10 degrees. and, the third range may mean a range in which the temperature difference between night and day is 10 degrees or more.

In this case, as an example, the first clothing recommendation information includes information on collectible clothing for which the thickness of the clothing is less than the first value, and the second clothing recommendation information includes the first clothing recommendation information for which the thickness of the clothing is greater than or equal to the first value and greater than the first value. Information on collectible clothing belonging to a value less than 2 may be included, and the third clothing recommendation information may include information about collectible clothing belonging to a value greater than or equal to the second value. Here, for example, the first value may be 1 mm and the second value may be 3 mm, but is not limited thereto.

According to this, the apparatus 100 may generate and provide customized clothing recommendation information suitable for the corresponding level according to the level of night and day temperature difference information to the user.

As described above, whenever a user inputs an input request message, the history storage unit 170 may generate and store wearing clothing history data corresponding to the input request message. The owned clothing storage unit 180 may store information on owned clothing owned by the user. Since this has been described in detail above, a detailed description thereof will be omitted.

The clothing recommendation information generated by the generator 120 in the device 100 may be clothing recommendation information related to attire information (coordination information) generated based on at least one of the clothes owned by the user. . However, the present invention is not limited thereto, and the clothing recommendation information generated by the generator 120 refers to fashion products sold and traded on an online shopping mall to be described later (this means various fashion-related products including clothing, fashion accessories, cosmetics, etc.) ) may be clothing recommendation information related to coordination information generated based on at least one fashion product.

As described above, in order to solve the problem that the conventional SNS providing technology has a risk of personal information exposure and a limitation in expressing information for sharing daily life, the device 100 displays the generated user avatar on a real image. By displaying in , it is possible to provide a function for obtaining an augmented reality image in which the user avatar is reflected.

In addition, the apparatus 100 displays the clothing recommendation information generated in consideration of the sensible temperature information that the user actually feels, and displays it on the screen of the user terminal, so that each user experiences the same under the same weather temperature condition. Considering that the relative sensible temperature (feeling) is different, it is possible to provide a function of providing (recommending) more useful customized clothing recommendation information customized to each user. In addition, the device 100 applies clothing corresponding to the clothing recommendation information generated in consideration of the sensible temperature information actually felt by the user to the user avatar and displays it on the screen to obtain an augmented reality image, thereby replacing itself. As the user avatar to express, it may be possible to apply (express) the user avatar in which the user's sensory temperature information or the user's actual emotions are reflected. As the present device 100 enables the user avatar to express himself in a more realistic way, the risk of personal information exposure can be reduced and his or her individuality can be more strongly expressed.

In addition, according to the simulation unit 190, which will be described later, the device 100 reduces the failure rate of the purchaser's purchase of the product and makes it possible to purchase it effectively when purchasing a fashion product (fashion product) sold on an online shopping mall. A simulation function can be provided. That is, by providing the simulation unit 190 , the device 100 provides the simulation unit 190 , so that when the product (product) is purchased and received on the online shopping mall, the feeling/size of the product viewed on the shopping mall is the feeling/size of the actual product. It is possible to provide a fashion styling simulation function that can solve problems in which product purchase satisfaction is not high and exchanges/refunds occur frequently due to differences in product purchases. The description of the simulation unit 190 may be more easily understood with reference to FIG. 6 .

6 is a schematic block diagram for explaining the simulation unit 190 included in the apparatus 100 for providing a social network service using augmented reality according to an embodiment of the present application.

Hereinafter, in the description with reference to FIG. 6, the configuration of the present system 1000, the present apparatus 100, the user terminal 1, and the network 2 shown in FIG. The system 1000 , the device 100 , the first user terminal 1 , and the network 2 may have the same configuration. Accordingly, even if omitted below, the contents described with reference to FIG. 1 may be equally applied to the contents described with reference to FIG. 6 .

Referring to FIG. 6 , the apparatus 100 may additionally include a simulation unit 270 in addition to the components included in the apparatus 100 illustrated in FIG. 1 .

The simulation unit 270 generates an avatar in consideration of the inputted body characteristic information when the user's (first user) body characteristic information is input, and relates to a desired wearing fashion product to which a wearing simulation is desired among a plurality of fashion products. When a user selection is made, an image of the product wearing avatar in which the selected desired fashion product is worn may be generated and displayed on the screen. A more detailed description of this is as follows.

The user terminal 1 may refer to the above-described first user terminal 1 as an example. However, the present invention is not limited thereto, and as described above, the description of the user terminal 1 is the same in the description of each of the plurality of user terminals 1 , 1 ', ... even if omitted below. can be applied.

The user terminal 1 may receive, for example, the user's body characteristic information from the user, and may provide the received body characteristic information to the input unit 10 of the simulation unit 270 . Accordingly, the input unit 10 may receive the user's body characteristic information from the user through the user terminal 1 . In addition, the user terminal 1 may control (remotely control) the operation of each unit in the apparatus 100 including the simulation unit 270 .

The simulation unit 270 may include an input unit 10 , an avatar generation unit 20 , a simulation control unit 30 , an identification unit 40 , and an evaluation unit 50 .

The input unit 10 may receive body characteristic information, which is information related to the user's body characteristics, from a user (user terminal). The input unit 10 may receive body characteristic information through the user terminal 1 or through a screen provided in the device 100 itself.

Here, the body characteristic information may include, for example, at least one of the user's body type, face shape, and skin tone. In this case, the information about the user's body type may mean information about the shape (shape, shape) of the user's body. However, the present invention is not limited thereto, and the information about the user's body type may include various types of information about the user's body size, such as the user's height, weight, foot size, waist size, and chest size. The body characteristic information may refer to all information necessary for generating a 3D avatar (3D model) resembling a user.

For example, the input unit 10 may directly receive the user's body characteristic information through typing from the user through the user terminal 1 .

As another example, the input unit 10 may receive the user's body characteristic information by extracting the user's body characteristic information from the user photographed image obtained from the user terminal 1 . As a specific example, in order to input body characteristic information, the user may photograph himself/herself so that the body characteristic information can be extracted using, for example, the camera of the user terminal 1 . At this time, shooting so that the user's body characteristic information can be extracted means the user's body part (or at least part of the user's body part) so that the user's body type (body shape), face shape, and skin tone appear. part) may mean photographing the front, side, etc., but is not limited thereto.

When a user-captured image, which is an image taken by the user himself/herself, is acquired through the user terminal 1, the input unit 10 performs image analysis on the acquired user-captured image to extract body characteristic information from the user-captured image can do. The input unit 10 may receive body characteristic information extracted from the user's photographed image as the user's body characteristic information. Here, various image analysis algorithms that are known or developed in the future may be applied to image analysis.

The avatar generator 20 may generate an avatar in consideration of the body characteristic information received from the input unit 10 .

In this case, the avatar generated by the avatar generating unit 20 may be the same avatar as the user avatar generated by the above-described avatar managing unit 210 . Accordingly, even if omitted below, the description of the user avatar generated by the avatar manager 210 may be equally applied to the description of the avatar (user avatar) generated by the avatar generator 20 , , and vice versa.

The avatar generated by the avatar generator 20 may be a three-dimensional avatar generated in consideration of at least one of a user's body type, face shape, and skin tone received as body characteristic information. The avatar generated by the avatar generating unit 20 may preferably be a 3D avatar generated by reflecting all body characteristic information received from the input unit 10 .

The avatar generator 20 may model and generate an avatar having a shape very similar to (similar to) a user's in three dimensions, based on the body characteristic information. The avatar generator 20 may generate an avatar having a body shape resembling the user himself/herself.

The simulation control unit 30 generates an image of a product wearing avatar in which the desired fashion product selected by the user is worn when a user selects a desired fashion product to be worn for which a wearing simulation is to be applied among a plurality of fashion products. can be displayed in Here, the product wearing avatar may mean an avatar in which the desired fashion product selected by the user is worn on the avatar created by the avatar generating unit 20 (the desired fashion product is worn). have.

The simulation control unit 30 may display the generated image of the product wearing avatar as an example on a screen provided in the device 100 itself, and as another example, may be displayed on the screen of the user terminal 1 .

As used herein, the term “fashion product” may refer to, for example, various fashion-related products sold and traded on an online shopping mall. Fashion products may include, for example, clothing, fashion accessories, cosmetics, and the like. Clothing may include various clothes such as tops, bottoms, dresses, and underwear, and fashion miscellaneous goods may include accessories (accessories), shoes, handbags, wallets, belts, hats, and the like. This example is only one example for helping understanding of the present application, and is not limited thereto, and the fashion product may include (consider) various articles, articles, etc. related to fashion, styling, appearance decoration, and the like. In the present specification, a fashion product may be referred to as a fashion item, a fashion product, or the like.

For example, before purchasing a specific fashion product among fashion products sold (traded) on an online shopping mall, the user may check whether the specific fashion product suits him well, how the fit looks when he wears it, width, length, etc. In order to confirm (simulation) whether the specific fashion product is tightened (tight part) when the specific fashion product is worn, the specific fashion product desired to be confirmed (simulated) may be selected as the desired fashion product to be worn.

That is, the user may select, as the desired fashion product, a fashion product that the user desires to perform a wearing simulation on the avatar generated by the avatar generating unit 20 among fashion products sold on the online shopping mall.

For example, a first fashion product (eg, top-short sleeve), a second fashion product (eg, bottom-shorts), and a third fashion product (eg, bottom-skirt) are sold on the online shopping mall as a plurality of fashion products. let's say it's happening In addition, if the user is interested in the first fashion product and wants to simulate the appearance when the user actually wears the first fashion product, the image of the first fashion product applied to the avatar generated by the avatar generating unit 20 is To view the image, the user may select the first fashion product as a desired fashion product to be worn.

In this case, in response to a user selection (user selection input) for a first fashion product among a plurality of fashion products, the simulation control unit 30 may synthesize a first fashion product, which is a desired fashion product, and a pre-generated avatar with each other. In this way, an image of the product wearing avatar in which the first fashion product is worn may be generated. In this case, the product wearing avatar means an avatar in which the first fashion product is worn on the avatar generated by the avatar generating unit 20 , and may refer to an avatar in which the first fashion product is worn.

The simulation controller 30 may display the generated image of the product wearing avatar on the screen. The user can virtually check the appearance of the first fashion product when the first fashion product is actually worn through the product wearing avatar displayed on the screen.

Thereafter, the simulation controller 30 may perform a simulation of changing the posture of the product wearing avatar. In this case, the posture of the product wearing avatar may be changed by, for example, a user's real-time posture change manipulation control, and as another example, may be automatically changed to a plurality of preset types of changing postures.

In this case, changing the posture of the product wearing avatar may mean changing the posture to a static state, but is not limited thereto, and is understood as a broad concept including all posture changes to a dynamic state. It may be desirable to be

Illustratively, the basic posture of the product wearing avatar may be an upright posture, and the changed posture may include a sitting posture on a chair at various heights, a squatting posture, a walking posture, a running posture, and the like, as well as a walking posture. , jumping, and repeating sitting and standing.

The simulation control unit 30 considers the product attribute information of the desired fashion product to be worn by the user, and displays the image of the posture change avatar reflecting the changed desired fashion product to be worn in real time, in which the state is changed by changing the posture of the product wearing avatar. can be created and displayed on the screen. That is, in response to the change in the posture of the product wearing avatar, the simulation control unit 30 reflects the change in the state of the desired fashion product to be worn (for example, a change in clothes stretched, tightened, tightened, etc.) An image of an avatar whose posture has been changed (posture change avatar) may be displayed on the screen in real time.

Here, the product attribute information of the desired fashion product to be worn may include information on at least one of dimensions, material, elasticity, and see-through level. For example, if the desired fashion product is a short-sleeved top, the size information may include not only size information such as S, M, L, but also information such as gun length, chest section, shoulder width, and sleeve length. , but is not limited thereto.

Product attribute information of the desired fashion product to be worn may be obtained, for example, by crawling (collecting) information disclosed on the homepage of an online shopping mall where the desired fashion product to be worn is sold.

According to this, when a user selects a desired fashion product from among a plurality of fashion products, the simulation control unit 30 acquires product attribute information on the desired fashion product selected by the user through crawling, and then obtains the product obtained An image of the posture-changed avatar to which the desired fashion product to be worn is applied may be displayed on the screen in real time, in which the state is changed as the posture of the product wearing avatar is changed in consideration of the attribute information. That is, the posture change avatar displayed on the screen in real time according to the posture change of the product wearing avatar means an avatar whose posture is changed (reflected) to which the state (shape, shape, etc.) of the desired fashion product to be worn is changed due to the posture change. can

The identification unit 40 may identify, on the posture change avatar displayed on the screen, a region of a tightening portion in which a tightening strength value greater than or equal to a preset tightening strength value appears by the changed (state change) desired fashion product. Here, the tightening strength value is a value indicating the level (degree) to be tightened, and may be referred to as a tightness degree value or the like.

The simulation control unit 30 may display the region of the tightening portion identified by the identification unit 40 on the image of the posture change avatar.

In the example of the present disclosure, the simulation controller 30 has illustrated that the region of the tightened portion identified on the posture change avatar is displayed on the posture change avatar, but the present disclosure is not limited thereto. The identification unit 40 may identify the region of the tightened portion on the product wearing avatar, and accordingly, the simulation controller 30 may display the region of the tightened portion identified on the product wearing avatar on the product wearing avatar.

The simulation controller 30 may change the display form of the region of the tightening portion displayed on the image of the posture change avatar according to the level (degree) of the tightening strength equal to or greater than the preset tightening strength value. At this time, the simulation control unit 30 determines that the region of the tightening portion having a high level of tightening strength among the regions of the tightening portion identified on the posture change avatar by the identification unit 40 shows a relatively low level of tightening strength. It can be controlled to display in a form that is easier to recognize or identify in contrast to the region of the tightening part.

In other words, in the simulation control unit 30, the tightening portion identified by the identification unit 40 includes the first tightening portion and the second tightening portion, and the tightening strength value of the first tightening portion is the tightening strength of the second tightening portion. If the value is greater than the value, the image of the posture change avatar (or on the product wearing avatar) may be controlled so that the region of the first tightened portion is displayed in a form that is easier to recognize or identify than the region of the second tightened portion.

Here, displaying the region of the tightening portion in a form that is easy to recognize or identify means that the display of the region of the tightening portion is visually emphasized, expanded in size, accompanied by an auditory effect, and a dynamic animation form It may mean displaying at least one of This can be more easily understood with reference to FIG. 7 .

7 is an example in which the region of the tightening portion identified by the identification unit 40 in the simulation unit 190 of the apparatus 100 for providing a social network service using augmented reality according to an embodiment of the present application is displayed on the screen. It is a drawing for explanation.

In FIG. 7 , as an example, as the identification unit 40 identifies the region of the tightened portion on the product wearing avatar 5 , the simulation controller 30 sets the region of the tightened portion identified by the identification unit 40 to the product wearing avatar 5 . An example of the case of displaying on the video of (5) is shown. However, the present invention is not limited thereto, and as described above, the simulation controller 30 may display the region of the tightened portion on the posture change avatar identified by the identification unit 40 on the posture change avatar.

In the description of the region identification and display of the tightening portion in the description to be described later, the description of the product wearing avatar 5 may be equally applied to the description of the posture change avatar, even if omitted below.

Referring to FIG. 7 , the simulation controller 30 may display an image of the product wearing avatar 5 in which the desired fashion product is worn, selected by the user, on the screen. In this case, the product wearing avatar 5 may refer to a state in which the desired fashion product selected by the user is worn on the three-dimensional avatar created in consideration of the user's body characteristics.

A plurality of desired fashion products selected by the user include, for example, a first desired fashion product 11 that is a top-nash, a second desired fashion product 12 that is a bottom-shorts, and a third desired fashion product that is a sock ( 13) is included. In this case, the simulation controller 30 converts the avatar (the avatar generated by the avatar generating unit) in which the first desired fashion product 11 to the third desired fashion product 13 to be worn to the product wearing avatar 5 . This can be displayed as an image on the screen.

In this case, the identification unit 40 may identify, on the product wearing avatar 5 , the region of the tightening portion in which the tightening strength value greater than or equal to the preset tightening strength value appears by the worn desired fashion product 11 , 12 , 13 . have. In this case, it is assumed that two parts including the first tightening portion 21 and the second tightening portion 22 are identified as the region of the tightening portion in which the tightening strength value greater than or equal to the preset tightening strength value appears by the identification unit 40 .

At this time, when the tightening strength value of the first tightening portion 21 is higher than the tightening strength value of the second tightening portion 22, the simulation control unit 30 performs the second on the image of the product wearing avatar 5 displayed on the screen. The region of the first tightening portion 21 may be displayed in a form that is easier to recognize or identify than the region of the second tightening portion 22 .

For example, the simulation controller 30 may display the area of the first tightening portion 21 in red and the area of the second tightening portion 21 in yellow. The simulation control unit 30 may display, for example, in the order of colors such as red, main, no, super wave, male, and bo as the tightening strength value gradually decreases based on the largest value. That is, the region indicated by the highest tightening strength value may be displayed in red, and the region indicated by the lowest tightening strength value may be displayed in purple.

As another example, the simulation control unit 30 may display the display size of the region of the first tightening portion 21 larger than the display size of the region of the second tightening portion 21 . As another example, the simulation control unit 30 may display the region of the first tightening portion 21 in bold and display the region of the second tightening portion 21 without bold.

As another example, the simulation controller 30 may display the region of the first tightening portion 21 in a blinking form, while displaying the region of the second tightening portion 21 in a static form. In this way, the simulation control unit 30 displays the region of the first tightening portion 21 in a blinking form (ie, blinking), thereby allowing the user to change the region of the first tightening portion 21 to the second tightening portion ( 22) can be provided so that it can be intuitively recognized that it is a part that is tightened relatively more compared to the area of 22).

That is, when the simulation unit 270 displays the region of the tightening portion on the posture change avatar or the product wearing avatar 5 (or on the image of the corresponding avatar), the region in which the tightening strength value is relatively higher is tightened. By displaying in a form that is easy to recognize or identify compared to an area where the intensity value is low, the user can intuitively recognize (confirm) in advance the area (location) where tightening is expected when the user purchases the desired fashion product and actually wears it. can be provided so that

The simulation unit 270 determines whether or not to purchase the desired fashion product to be worn before the user actually purchases the desired fashion product to be worn, to determine whether to purchase a different size compared to the desired fashion product to be worn; Alternatively, a decision on whether to purchase a different type of fashion product different from the desired fashion product to be worn may be made more quickly and easily, and the probability of failure of purchasing the fashion product may be reduced.

In addition, when it is determined that the number of regions of the tightening portion identified by the identification unit 40 is equal to or greater than a preset number, the simulation control unit 30 may be configured to exceed the desired wearing fashion product in which the region of the tightening portion is greater than or equal to the preset number. Information on the same type of fashion product having a larger dimension may be displayed on one area of the screen for recommendation as a recommended fashion product.

Here, the preset number may be, for example, three, but is not limited thereto, and the number may be variously changed and set (applied).

In the present application, a region of the screen on which information of the same type of fashion product is recommended may be referred to as the first region of the screen for convenience of explanation. In the first area of the screen, information on the same type of fashion product (including information on the same type of fashion product with a larger or smaller size) may be displayed, but the present invention is not limited thereto, and information on fashion products different from the same may be displayed. That is, the first area of the screen may mean an area in which a recommended fashion product for recommendation to a user is displayed among various fashion products including information on the same type of fashion product.

In other words, in an example of the present application, information of the same type of fashion product having a dimension larger than that of the desired fashion product to be worn as a recommended fashion product is displayed in the first area of the screen, but the present disclosure is not limited thereto, and the first area is not limited thereto. Regarding the desired fashion product to be worn, in which the number of areas of the tightening area is more than the preset number, the desired fashion product to be worn is of the same type but larger in size (which is the same in color, shape, pattern, etc. It means a different type of desired fashion product), a similar type but smaller in size, and a different type of desired fashion product (this means that at least one of a different color, shape and pattern) b) may be displayed as recommended fashion products.

For example, the posture change avatar (or the product wearing avatar) is wearing a plurality of desired fashion products, and the identification unit 40 determines the region of the tightening part in any one of the plurality of desired fashion products to be worn. Assume that it is identified that it appears more than a set number. In this case, the simulation control unit 30 determines that the desired fashion product to be worn (ie, any one desired fashion product) in which the region of the tightening part appears greater than or equal to a preset number is small for the user (that is, the user desires to wear the corresponding fashion product). When the product is actually purchased and worn, it is judged that it may be small), and information about the same type of fashion product having a size (size) that is one (or more than two) larger than the desired fashion product is recommended to the user as a recommended fashion product. The recommendation may be displayed in one area (the first area) of the screen.

At this time, the simulation control unit 30, when the region of the tightening part more than a preset number for the desired fashion product to be worn is identified (for example, when 5 regions of the tightening part are identified), the area of the identified tightening part is Decide whether to recommend a fashion product with a size difference from the desired fashion product to be worn in consideration of at least one of the number, the tightening strength value for each region of the identified tightening portion, and the location of the identified tightening portion region can

Exemplarily, when the desired fashion product to be worn is size M, and the number of regions of the tightened portion identified in the desired fashion product to be worn is 8 or more, the simulation control unit 30 controls the size of the desired fashion product by two steps (XL size) compared to the desired fashion product. ) of the same type of fashion product may be displayed in the first area. If the desired fashion product to be worn is M size, and the number of areas of tightening parts identified in the desired fashion product to be worn is 3 or more but less than 8, the simulation control unit 30 controls the size by one step (1 step) larger than the desired fashion product to be worn ( That is, information on fashion products of the same type having size S) may be displayed in the first area.

The information of the same type of fashion product displayed on the first area of the screen may be an image of the same type of fashion product, but is not limited thereto, and may include at least one of product attribute information of the same type of fashion product.

In the above-described example of the present application, as the identification unit 40 identifies the region of the tightening portion in which the tightening strength value greater than or equal to the preset tightening strength value (which may be referred to as a preset upper limit of tightening strength) is identified, the result is Accordingly, the case in which the simulation control unit 30 recommends the same type of fashion product having a larger dimension has been exemplified, but the present invention is not limited thereto. As another example, the identification unit 40 identifies the region of the tightening portion in which the tightening strength value corresponding to less than the preset tightening strength lower limit appears, and according to the result, the simulation control unit 30 provides the desired fashion product to be worn to the user. It is determined that the size is too large and the same type of fashion product having a size smaller than the desired fashion product to be worn by one (or two or more steps) may be displayed as a recommended fashion product on the first area of the screen.

The simulation control unit 30 is configured to display, as a recommended fashion product, information (eg, an image) of a fashion product of the same type having a dimension larger than that of a desired fashion product to be worn on the first area of the screen (eg, an image), then the information of the displayed fashion product When a user input is made to , an image of an avatar to which a product of the same type is worn is generated (regenerated) and displayed on the screen in response to the user input.

That is, when a user input for a recommended fashion product is made in the first area of the screen, the simulation control unit 30 regenerates an image of an avatar to which the recommended fashion product is worn in real time and is displayed on the screen in response. can be displayed

The simulation unit 270 shows, in real time, the appearance of the avatar in which the user's selected fashion product (the user's desired wearable fashion product, the user's selected recommended fashion product) is worn through the product wearing avatar, in which case the fashion product is displayed. By displaying on the screen the location of the part that is too tight or too loose by the fashion product worn on the avatar that models the user's body type, the avatar created in consideration of the user's body characteristics can be predicted when a specific fashion product is actually worn. It can be provided realistically.

In addition, the simulation unit 270 can change the posture of the product wearing avatar and display the region of the tightening part that appears whenever the posture is changed in real time, so that the posture can be adjusted dynamically by moving the desired fashion product to be worn in an upright posture as well as simply when the desired fashion product is worn in an upright posture. Even when a change is made, the user can intuitively recognize the expected tightening part, so it can help a decision to purchase the desired fashion product to be worn more decisively, and failure to purchase a fashion product in an online shopping mall The probability can be effectively reduced.

The evaluation unit 50 may perform styling evaluation on the product wearing avatar in the image (or the posture change avatar image) of the product wearing avatar displayed on the screen by the simulation control unit 30 .

The simulation control unit 30 may determine whether to display information of a recommended fashion product in one area of the screen according to a result of the styling evaluation by the evaluation unit 50 . In this case, one region of the screen in which information of recommended fashion products is displayed based on the styling evaluation result may be referred to as a second region for convenience of description in the following description.

The evaluation unit 50 calculates evaluation scores for a plurality of evaluation items when performing styling evaluation, and scores evaluation items whose evaluation scores do not satisfy a preset evaluation score among the plurality of evaluation items based on the calculated evaluation scores It can be identified as an under-evaluation item. In particular, the evaluation unit 50 may calculate a styling evaluation score for the user avatar image, which will be described later, which may mean evaluation scores (total evaluation scores) for a plurality of evaluation items in relation to the user avatar image. have.

Thereafter, the simulation control unit 30 responds to the identification of the low-scoring evaluation item, and replaces the desired fashion product related to the low-scoring evaluation item among the desired fashion products worn on the product wearing avatar with another fashion product. Information on recommended fashion products may be displayed in one area (second area) of the screen.

Here, the plurality of evaluation items include a first evaluation item regarding the degree of matching between desired fashion products worn on the product wearing avatar and a matching degree between the skin tone of the product wearing avatar and the desired fashion product worn on the product wearing avatar. A second evaluation item may be included. However, the present invention is not limited thereto, and the plurality of evaluation items includes the degree of matching between the avatar and the fashion product worn on the avatar in relation to design (eg, a pattern or pattern of a fashion product), color (color) combination, etc. , any item that evaluates the degree of harmony between fashion products is applicable.

For example, according to the first evaluation item, when shoes and a top are worn as a desired fashion product to be worn on the product wearing avatar, whether the shoes and the top match each other (the degree of matching with each other is evaluated) may be evaluated. Also, according to the first evaluation item, when, for example, a top-cotton tee and a bottom-skirt are worn on the product wearing avatar, whether the top and bottom match each other may be evaluated.

Also, according to the second evaluation item, as an example, whether the skin tone of the avatar (eg, the skin color of the face) and the color of the top, which is a desired fashion product, match each other may be evaluated. In addition, according to the second evaluation item, it may be evaluated whether the skin tone or face shape of the avatar and the pattern pattern of the desired fashion product to be worn (for example, a stripe pattern or a check pattern) match each other.

The evaluator 50 may calculate an evaluation score for the product wearing avatar (ie, evaluation scores for a plurality of evaluation items) when evaluating the styling of the product wearing avatar, wherein the evaluation score for the product wearing avatar is a plurality of evaluation points. It may be calculated in consideration of the evaluation score of others evaluated by other users.

For example, in order for the user to evaluate the overall styling of the avatar (product wearing avatar, posture change avatar) in which the desired fashion product is worn, which is a fashion product that the user wants to purchase, in order to be evaluated by other users (ie, styling by other users). In order to receive evaluation), for example, an evaluation request signal for an image of an avatar (product wearing avatar, posture change avatar) generated by the simulation controller 30 and displayed on the screen may be input to the simulation unit 270 . In the following description, an image of an avatar (product wearing avatar, posture change avatar) generated by the simulation controller 30 and displayed on the screen is referred to as a user avatar image for convenience of explanation.

In this case, the evaluation request signal is an avatar image (ie, user avatar) generated by the simulation controller 30 in response to a user input to the evaluation request button when a user input is made to the evaluation request button provided in one area of the screen. image) may mean a signal for uploading to a social network service (Social Network Services/Sites, SNS).

When the user avatar image is uploaded to the SNS according to the evaluation request signal, users of the SNS (ie, other users) to which the image has been uploaded may perform styling evaluation related to a plurality of evaluation items on the user avatar image. can Users of the SNS (ie, other users) may provide (or rate) an evaluation score in relation to a plurality of evaluation items for the user avatar image, and the score evaluated by other users is referred to as the evaluation score of others. can be

The evaluation unit 50 considers at least one score (ie, at least one evaluation score of others) evaluated by other users for the user avatar image and a styling evaluation score (ie, the user avatar image and the user's avatar image). In this regard, a total evaluation score for a plurality of evaluation items) may be calculated. In this case, the styling evaluation score related to the user avatar image may be calculated as, for example, an average value of evaluation scores of others evaluated for the user avatar image, but is not limited thereto.

The evaluation unit 50 may identify an evaluation item in which an evaluation score does not satisfy a preset evaluation score among a plurality of evaluation items in the styling evaluation score related to the user avatar image as an under-score evaluation item. Here, as an example, it is assumed that an under-scored evaluation item is identified as a second evaluation item.

In this case, the simulation controller 30 determines that the skin tone of the avatar wearing the product in the user avatar image and the desired fashion product worn by the avatar wearing the product do not match each other (ie, the desired fashion product that the user wants to purchase). It is determined that it does not match the user's skin tone), and information on a recommended fashion product may be displayed on the second area of the screen to replace the desired fashion product to be worn related to the under-scored evaluation item with another fashion product. In this case, the recommended fashion product displayed in the second area of the screen may be referred to as an alternative recommended fashion product for replacing the desired fashion product to be worn related to the low-scoring evaluation item.

In addition, the evaluation unit 50 calculates the styling evaluation score related to the user avatar image (that is, the total evaluation score for a plurality of evaluation items in relation to the user avatar image), in addition to using the evaluation score of others, another example The styling evaluation score can be calculated using the deep learning model trained with Since examples of the types of deep learning models have been described above, overlapping descriptions will be omitted below.

The deep learning model, for example, is a learned neural network model that learns by inputting body characteristic information (face shape, skin tone, size, etc.) of a plurality of users and using fashion products matching each of the body characteristic information of a plurality of users as an output value can mean

The simulation controller 30 may provide (display) an image of an alternative recommended fashion product, a list of alternative recommended fashion products, etc. as information (information on an alternative recommended fashion product) displayed on the second area of the recommended fashion product.

At this time, in the list included in the recommended fashion products (recommended alternative fashion products) displayed on the second area of the screen, fashion products of the same type that differ only in color from the desired fashion product related to the under-scoring evaluation item and wearing related to the under-scoring evaluation item Other heterogeneous fashion products with similar performance (shape, material, elasticity, see-through, etc.) to the desired fashion product may be included. In this case, the different types of fashion products may include fashion products with better reviews (evaluation), lower prices, or higher quality compared to the desired fashion products related to the low-scoring evaluation item.

According to this, the simulation unit 270 is a fashion product evaluated as unsuitable for the user by the evaluation unit 50 among the fashion products that the user desires to purchase (that is, the fashion product that the user wants to wear (ie, desired to wear in relation to the low-scoring evaluation item). In providing information (for example, a list) of alternative recommended fashion products (recommended alternative fashion products) to replace fashion products), It is possible to provide (recommend) a fashion product that is different from the one with good performance but is included in the list of alternative recommended fashion products. The simulation unit 270 allows the user to rationally compare/select products when replacing (replacement) the desired fashion product to wear related to the evaluation item with a low score, and to make replacement/management easier and more efficient. can support

Also, the simulation controller 30 may display a purchase button for purchasing a desired fashion product included in the image of the product wearing avatar on one area of the screen. In this case, in the description to be described later, the one region of the screen where the purchase button is made may be referred to as a third region of the screen differently.

When a user input is made to the purchase button displayed on the third area of the screen, the simulation control unit 30 may transmit a purchase request signal for purchasing a desired fashion product worn on the product wearing avatar displayed on the screen.

Here, the purchase request signal is a signal provided to enable automatic payment of the desired fashion product to be worn (or automatic delivery through automatic payment is possible) in connection with an online shopping mall that sells the desired fashion product to be worn and/or the desired fashion to be worn It may be a signal for providing a message notification on the screen of the user terminal 1 associated with the device 100 in order to encourage an order for a product.

The simulation unit 270 transmits a purchase request signal in response to a user input for a purchase button so that the user can simultaneously purchase fashion products desired to be purchased after viewing a wearing simulation for the desired fashion product. can be provided, so that the user's convenience in purchasing fashion products can be improved.

In addition, the present system 1000 may include a front display device 3 interlocked with the present device 100 . The device 100 and the front display device 3 may be linked (connected) through the network 2 , and since the description of the network 2 has been described above, the overlapping description will be omitted below.

The simulation control unit 30 may control the operation of the front display device 3 interlocked with the present device 100 . When it is determined that a user input has been made to the sensory data providing button displayed in one area (eg, the fourth area) of the screen, the simulation control unit 30 responds to the user's input to the sensory data providing button of the desired fashion product to be worn. The operation of the front display device 3 may be controlled so that fashion product sensory data corresponding to a desired fashion product to be worn in consideration of product attribute information is provided to the front of the front display device 3 .

Here, the fashion product realization data refers to data that implements a desired fashion product as a three-dimensional image ( That is, it may mean 3D stereoscopic image data). That is, the sensory data of the fashion product may mean 3D stereoscopic image data of the desired fashion product to be worn, which is implemented in a size corresponding to the actual (actual) size of the desired fashion product. In other words, the fashion product sensory data may mean a three-dimensional fashion product (fashion product of a three-dimensional shape) corresponding to a desired fashion product to be worn shown through a beam irradiated into the sky by the front display device 3 . .

For example, it is assumed that information on the total length, chest section, shoulder width, and sleeve length of the desired fashion product to be worn is obtained through crawling as product attribute information of the desired fashion product to be worn. At this time, when a user input for providing the sensory data of the fashion product for the desired fashion product to be worn is made (that is, when a user input is made for the sensory data providing button), the simulation control unit 30 controls the front display device 3 It is possible to control the operation of the front display device 3 so that the fashion product sensory data is provided in the front through the . At this time, the provided fashion product realism data is data obtained by imaging the desired fashion product to be worn in a three-dimensional image according to the relevant dimensions based on the information on the total length, chest section, shoulder width, and sleeve length of the desired fashion product. can

The simulation control unit 30 may be linked (connected) with the front display device 3 through the network 2 to control the operation of the front display device 3 . A Head Up Display (HUD) 3 can provide real-world data of fashion products in front of the front display device 3 (or in the air in front, in the air in front) through beam irradiation. have.

The front display device 3 may provide the sensory data of the fashion product to the front of the front display device 3 under the control of the simulation controller 30 . The front display device 3 may focus on the front of the front display device 3 through a beam, and from this, an image (image) corresponding to the sensory data of a fashion product at a position corresponding to the focus can be displayed.

In other words, when a user input to the sensory data providing button is made, the front display device 3 scatters the image in front of the front display device 3 through a beam in response to the control of the simulation controller 30 to produce a fashion product. The sensory data may be provided on the sky in front of the front display device 3 .

8 is a diagram illustrating an example of sensory data for a fashion product provided by the front display device 3 interworking with the social network service providing device 100 using augmented reality according to an embodiment of the present application.

Referring to FIG. 8 , in response to a user input for the sensory data providing button, the simulation control unit 30 displays the sensory data of a fashion product corresponding to a desired fashion product to be worn related to the sensory data providing button on the front of the front display device 3 . It is possible to control the operation of the front display device 3 so as to be provided in the sky.

In general, in online shopping malls that sell and trade fashion products, the display of information indicating the size of fashion products (eg, gun length, chest section, shoulder width, and sleeve length, etc.) mm, etc.). For this reason, it is difficult for the user to easily estimate (judgment) what the size of the fashion product sold on the online shopping mall is when actually receiving it.

That is, in the prior art, because the user could estimate the size of the corresponding fashion product only based on the numerical value displayed on the online shopping mall, when the actual fashion product was actually received, the actual size of the fashion product was greater than the size of the fashion product expected by the user (the user himself). There have been frequent cases where the size of the fashion product is larger or smaller, and in this case, the user has to exchange or refund the fashion product with a different size again.

Accordingly, the simulation unit 270 may provide the sensory data of the fashion product, which is a 3D stereoscopic image having a size corresponding to the actual size of the sensory data of the fashion product, to the user through the front display device 3 . This simulation unit 270 receives fashion product real-world data before purchasing/paying for a fashion product that the user desires to purchase (fashion product desired to be worn), so that The actual size (actual size) can be intuitively recognized and provided so that you can experience it.

In particular, the simulation controller 30 may control the movement of the front display device 3 so that the sensory data of the fashion product provided through the front display device 3 is projected onto the user's body. To this end, although not shown in the drawings, the present system 1000 may include an image sensor (not shown) and a movement control member (not shown) connected to the simulation unit 270 through a network.

Here, the image sensor (not shown) may mean a sensor that acquires a body image of a user who is a target of projection of sensory data of a fashion product. The image sensor (not shown) may be expressed differently as a camera, an image capturing sensor, or the like. The image sensor (not shown) may be, for example, at least one of an image sensor provided in the user terminal 1 , an image sensor provided in the front display device 3 , and an image sensor provided in the simulation unit 270 , which is limited thereto. it is not

A movement control member (not shown) may be coupled to the front display device 3 and control movement of the front display device 3 for left/right/up/down/front/rear. A movement control member (not shown) may control movement and angle of the front display device 3 . That is, the movement control member (not shown) may control the angle as well as the position of the front display device 3 providing the sensory data of the fashion product.

As an example, the simulation control unit 30 performs image analysis on a body image of a user (a user who is a target of projection of sensory data of a fashion product) obtained through an image sensor (not shown), and the user's partial region (eg, Head, neck, arm, wrist, upper body, lower body, ankle, foot, etc.) can be identified. In addition, the simulation control unit 30 may identify a classification type (eg, whether it is a top or bottom, whether it is an accessory such as a bracelet, a hat, etc.) of a fashion product to be worn corresponding to the sensory data of the fashion product.

Thereafter, the simulation control unit 30 determines the fashion product sensory data corresponding to the desired fashion product to be worn on the corresponding user's body part based on the identified classification type of the desired fashion product to be worn and the identified partial part of the user. By controlling the operation of the movement control site (not shown) to be projected on the front display device 3 can be moved.

Exemplarily, when the classification type of the desired fashion product to be worn is upper, the simulation controller 30 may control the desired wear fashion identified from among the user's partial regions in which fashion product sensory data corresponding to the desired fashion product is identified through image analysis. Moving the front display device 3 through the operation control of the movement control member (not shown) so as to be projected onto the upper body that is the body part corresponding to the classification type of the product (that is, the upper body that is the body part corresponding to the upper body). control can be performed. That is, the simulation control unit 30 may move the front display device 3 so that the fashion product sensory data corresponding to the desired fashion product (top) to be worn on the upper body of the user's body part is overlapped and projected. An operation of a movement control member (not shown) may be controlled for movement of the device 3 .

The device 100 may effectively reduce a failure probability of purchasing a desired fashion product by providing the fashion product sensory data through the simulation unit 270 .

Accordingly, the device 100 may be referred to as an apparatus for providing a social network service using augmented reality. In addition, the apparatus 100 generates clothing recommendation information, thereby providing a clothing recommendation device and a simulation unit 270 . ), as it is a device capable of simulating fashion styling, it may be referred to as a fashion styling simulation device or the like.

This device 100 may be implemented in the form of an application installed in the user terminal 1 as described above. The device 100 may be expressed as a personalized fashion platform application that respects individual diversity by the simulation unit 270 .

The simulation unit 270 may generate avatars of various body types resembling itself as an avatar in consideration of the user's body characteristic information through the avatar generating unit 20 , and may simulate a desired coordination on the generated avatar. According to the simulation unit 270 , the apparatus 100 may perform style presentation for each theme in consideration of the user's size. According to the simulation unit 270 , the device 100 may subscribe to and receive various fashion related news through SNS, for example.

According to the simulation unit 270, the device 100 can move to a shopping mall site (Internet shopping mall site) linked through an icon displayed for each category, for example, and provide fashion products sold on the moved shopping mall site to purchase. can do. In addition, the simulation unit 270 may transmit a purchase request signal for purchasing desired fashion products included in the image of the product wearing avatar in response to a user input for the purchase button, and through this, It can be provided to make the purchase easier and more convenient.

The simulation unit 270 may record a daily look by applying a fashion product such as clothes worn today to the avatar. To this end, although not shown in the drawing, the simulation unit 270 uses a first data storage unit (not shown) for acquiring and storing information about the collection fashion product owned by the user (information on the collection fashion product) and the daily look. It may include a second data storage unit (not shown) for recording (storing).

Here, the first data storage unit (not shown) may have the same configuration as, for example, the owned clothing storage unit 180 described above. Accordingly, even if omitted below, the description of the owned clothing storage unit 180 may be equally applied to the description of the first data storage unit (not shown).

Illustratively, the user selects a collectible fashion product corresponding to a fashion product actually worn by the user for going out as an example among the collectible fashion products stored in the first data storage unit (not shown) (herein, the selected collectible fashion product includes As an example, the above-described selected clothing to be worn may be included) and worn on the avatar generated by the avatar generating unit 20, and then the second data storage unit (not shown) using the product wearing avatar on which the collectible fashion product is worn as a daily look. time) can be recorded (stored).

As such, the second data storage unit (not shown) may store daily look information in which the user records clothes and the like when going out. In addition, the second data storage unit (not shown) may store an image (or an image of the posture change avatar) of the product wearing avatar, which is the subject of styling evaluation, as described above.

The apparatus 100 may have the following functions by the simulation unit 270 .

The device 100 uploads, for example, daily look information stored in the second data storage unit (not shown) and/or image information of an avatar wearing a product, which is the subject of styling evaluation, in connection with SNS, and for the uploaded information You can get reviews from other users.

Also, when a user input is made to the 'My Closet' icon menu displayed on one area (eg, the fifth area) of the screen for example, the device 100 may view fashion products that the user wishes to purchase or purchases at a glance. It can be displayed on the screen for viewing.

In addition, the device 100 searches for information on an acquired fashion product based on, for example, acquisition of a QR code (QR code) related to a fashion product and/or acquisition of a fashion product image photographed by the user terminal 1 . can be performed.

Also, in the case of an accessory, the device 100 may provide the actual size to be applied to the user's body as an AR model, for example. The device 100 provides a 3D stereoscopic image having a size corresponding to the actual size of a fashion product that a user desires to purchase among fashion products sold on an online shopping mall (ie, providing realistic data of a fashion product) By doing so, it is possible to provide a more realistic wearing simulation for a fashion product that the user desires to purchase (fashion product desired to be worn).

The device 100 can diversify a user's body type, face shape, size, etc. according to individual characteristics by using the avatar generated by the avatar generating unit 20 through the simulation unit 270 . It can provide a differentiator from the app.

The device 100 may provide technology that can help coordinate fashion products, purchase clothes, and the like, by supplementing the limitations of online shopping malls that cannot actually fit. The device 100 may provide various types of fashion products including clothes, accessories, and makeup elements so that they can be coordinated on the avatar generated by the avatar generator 20 . Also, the device 100 may provide items (fashion products) of various themes to be updated regularly so that new products (new fashion products) are not omitted.

In addition, the device 100 may be implemented so that the app itself has an SNS function. As an example, the device 100 may provide a function that allows another person (other user) on the SNS to store or recommend my coordinator (coordinate information uploaded and shared by the user on the SNS). In this case, when another user stores or recommends my co-ordination, the device 100 may provide points to other users. Conversely, when the user stores or recommends another user's coordination, the user may receive points from the device 100 . In this case, the points provided by the device 100 may be provided to enable purchase of fashion products sold/traded on social media and online shopping malls, for example, but is not limited thereto and other rewards (eg, exchange for real money, etc.) may be provided.

Also, when a new user (new user) is added through sharing the application, for example, the device 100 may pay points to the user (user) who has shared the application. Also, even when clothes are purchased through the app of the device 100 , the device 100 may provide points to the user (user) who purchased the clothes. Also, the device 100 may provide points as a reward to a user who has promoted an event and/or promotion for a specific fashion product.

Also, by collecting points provided by the user, the device 100 may allow the points to receive a discount when purchasing an actual fashion product or to enable the purchase of an actual fashion product.

The device 100 enables simulation of product wearing of a product (a fashion product desired to be worn) purchased by a buyer (a user wishing to purchase a fashion product) on an online shopping mall, thereby effectively reducing the consumer's burden of failure to purchase. . In addition, the device 100 may provide a convenient purchase by moving directly to a desired site because online shopping malls are concentrated at once. This device 100 can eliminate inconvenience and provide more convenience (convenience) when a consumer (buyer) purchases a fashion product.

The device 100 allows the user (the buyer) to more intuitively check the wearing fit of the fashion product by simulating how the fashion product is worn (pre-wearing) before the user (the buyer) purchases the fashion product sold in the online shopping mall. can provide

The apparatus 100 may help the user (buyer, consumer) to purchase a fashion product desired to be purchased more smoothly. In addition, the apparatus 100 may help the simulation unit 270 to shop with a low failure rate (reducing failure rate) of the purchase of fashion products by the user.

The device 100 may allow the simulation unit 270 to make a customized recommendation for a fashion coordination style to the user, and may provide a more realistic product wearing simulation function.

From a business need point of view, the device 100 may sell more clothes to increase sales of shopping malls to provide a smooth shopping service. Also, from the viewpoint of user needs, the device 100 can help to recommend and purchase clothes that best suit me at a low price.

The device 100 can solve the conventional problem that a fashion product sold on an online shopping mall needs to be exchanged/refunded because the size does not match when the user actually wears it, unlike the picture.

When the desired fashion product selected by the user from among the plurality of fashion products is cosmetics (eg, nail polish, BB cream, sunscreen, etc.), the device 100 generates an avatar image generated through the avatar generating unit 20 Wearing simulation (product wearing simulation) in which makeup (makeup) or nail art using the corresponding cosmetics is applied can be performed.

In addition, the device 100 may collect and display, for example, season-specific recommended fashion products pursued by a brand on one screen, or collect and display fashion products corresponding to the number one sales rate in each shopping mall on one screen. can

Also, the device 100 may provide a recommended coordination book category function, a function to check fashion coordination of application users, a communication function, a function to view similar fashion designs at a glance, and a function to compare prices of similar fashion products. Also, the device 100 may provide a function of recommending an outfit according to the taste of an application user, and a function of sharing a fashion coordination with users or with a friend.

At this time, to recommend/share a coordinator here means uploading an image (shape) of a product wearing avatar (or posture change avatar) in which a desired fashion product is worn by application of a wearing simulation, as an example, to SNS. It can mean an action such as recommending or sharing.

The device 100 provides a fashion styling coordination recommendation function, a clothing price adjustment function, a fashion style coordination simulation function, a communication function with other users (a communication function between users, a style sharing function), and a similar style clothes coordination style at once. It is possible to provide a function to check, a function to receive a discount when purchasing fashion products through points, and an image search function (a function to compare and analyze similar fashion products).

Also, in the device 100 , the avatar generator 20 may generate an avatar in consideration of body characteristic information (eg, body shape, etc.) for each individual.

The device 100 performs a wearing simulation (product wearing simulation) by the simulation unit 270 when the wearing fit (avatar wearing fit) of the fashion product shown through the avatar and the actual user purchase and wear the corresponding fashion product. It can be provided so that the worn fit (actual worn fit, actual coordinated appearance) of the wearer looks quite similar, thereby effectively reducing the probability of failure to purchase a fashion product on an online shopping mall.

In addition, the device 100 may provide the application itself to have an SNS function through association with the SNS.

The device 100 confirms the fit of the fashion products coordinated by the device 100 (ie, desired fashion products worn on the avatar generated by the avatar generating unit) and simultaneously purchases the corresponding fashion products at the same time can be provided to do so. Through this, the device 100 may improve the convenience of purchasing fashion products.

In addition, the device 100 may receive a recommendation for a style similar to or similar to the style (style of a fashion product) coordinated by the device 100 through the provision of the recommended fashion product, and through this, purchase/comparison of fashion products, etc. It can be provided to make this happen smoothly. The device 100 may provide a user with a satisfactory shopping purchase of fashion products, and may reduce the probability of failure.

In the past, users had to compare the price/performance of the fashion products provided by each online shopping mall individually. There were problems (inconvenient) such as failure to In contrast, by providing recommended fashion products, the device 100 allows the user to more effectively compare fashion products and to effectively search for and receive optimized fashion products desired by the user, such as fashion products at an appropriate price. have.

The device 100 may provide, for example, new news related to a fashion product uploaded on an application or SNS by other users, information on a coordinating recommendation, information on a recommended user, and the like, on the screen. Also, the device 100 may classify a plurality of fashion products into categories (eg, tops and bottoms, accessories, shoes, etc.) for example and provide the fashion products classified by each category as corresponding fashion products. It can be provided in conjunction with the information of the online shopping mall that sells them.

A plurality of fashion products may be displayed on the screen of the device 100 . In this case, the plurality of fashion products may refer to fashion products sold/traded on the online shopping mall. The device 100 may display, for example, a search window for searching for fashion products in one area of the screen. The user may input a keyword for searching for a desired fashion product in the search window displayed on the screen, and the device 100 displays a fashion product (matching fashion product) matching the input keyword among a plurality of fashion products on the screen. can be displayed Thereafter, when any one of the matching fashion products is selected, the simulation controller 30 displays the matching fashion product selected by the user on the avatar (meaning the avatar generated by the avatar generator) displayed on one area of the screen. An image of a product wearing avatar (ie, an avatar in which a wearing simulation of a matching fashion product selected with respect to the generated avatar is performed) may be displayed on the screen.

Through this, the device 100 allows the product wearing avatar displayed on the screen to visually check the appearance (fit of the matching fashion product worn) coordinated on the avatar modeled in a shape similar to the matching fashion product. can be provided to do so. In addition, the device 100 desires to purchase such a matching fashion product, and when a user input to the purchase button is made, the device 100 moves to an online shopping mall that sells the matching fashion product (that is, to sell the matching fashion product) control so that the screen of the online shopping mall is displayed) or transmit a purchase request signal enabling purchase of the corresponding matching fashion product.

Also, when a keyword for searching for a fashion product is input in the search window, the device 100 may control the screen display so that search terms related to the corresponding keyword are displayed together.

The device 100 provides a function to check my character on the home screen, a function to save after creating a character, a function to check a stored completed character, a function to move to the avatar list when pressing an avatar on the home screen, and an avatar from the stored avatar list function to delete by selecting , a function to move to the ranking list screen of shopping malls by selecting the icon on the home screen, a display function to check the shopping malls you wish to at a glance by pressing the Favorites button, a function to select and delete the shopping malls you wish to , a function that allows you to check the items (fashion products) you wished for on one screen in the shopping mall you wished can do.

In addition, the device 100 captures a fashion product with a camera and cuts the photographed image to perform a fashion product search function, a function to provide search results for related fashion products similar to the photographed image, and detailed information in a banner during shopping A function that allows you to shop faster because you can set shopping conditions carefully, a function to check the information of a desired fashion product in the shopping mall at a glance, and a function to wear the desired fashion product to your saved avatar (wearing product simulation). It may include a function to check fit, a function to check delivery/inquiry history, a function to check frequently asked questions, and the like, a login screen function, and the like. However, the various functions included in the above-described device 100 are merely examples for helping understanding of the present application, and are not limited thereto.

In other words, the device 100 compensates for the limitations of online shopping malls that cannot be fitted in reality to help purchase fashion products including outfits and clothes. technology is provided.

The device 100 can reduce the consumer's purchase failure burden by making it possible to simulate wearing the product purchased by the purchaser in the online shopping mall, and can make the purchase easier, and simulate wearing the product in advance to ensure the fit. can be provided for verification.

The device 100 enables movement and purchase to a linked shopping mall site through icons displayed by classification, records daily looks by applying the clothes to wear today to the avatar, and searches QR codes and images related to fashion products (products) This is possible, and in the case of accessories, the actual size to be applied to the body can be helped by an AR model.

The device 100 performs a simulation of changing the avatar posture, such as placing the avatar on a chair or raising the arm, after the avatar created in consideration of the user's actual body type is put on (after wearing) fashion products such as clothes. can do. In this case, the device 100 may display the position (position on the avatar) of the part where tightness, tightness, etc. occur through simulation. For example, the device 100 may display the position of the portion where the tightening occurs as the posture of the avatar is changed, such as shoes or clothing worn by the avatar.

In addition, the apparatus 100 may additionally provide information predicting the intensity of tightening felt by the user (intensity of tightness).

In addition, the device 100 provides a simulation of wearing a fashion product according to a change in posture for the avatar, so that when the avatar changes posture to sit on a chair, the avatar wears a seated posture on the avatar according to the posture change. The user can visually confirm the expected appearance in which the hem of the worn pants rises, or the expected appearance in which the thigh portion of the pants is tightened.

Also, when the avatar 100 is wearing a fashion product or changing a posture, when multiple tightening parts (several areas) are identified, the display color of the tightening parts displayed on the screen is different according to the tightening strength. can do. Also, for example, when the number of tightening parts is greater than or equal to a preset number, the device 100 may display a fashion product having a size larger than that of the corresponding fashion product worn on the avatar as a recommended fashion product in one area of the screen. . In this case, when a user selection input is made for any one of the recommended fashion products, the device 100 applies (wears) the selected one of the recommended fashion products to the avatar in real time (recommended fashion products). of wearing shot) can be provided by displaying it on the screen.

In addition, the device 100 may include a function to be evaluated by a number of other users for the fashion coordinated by the user based on the avatar generated in consideration of the user's skin tone, etc., through linkage with the SNS. can

In addition, the apparatus 100 may provide information on clothing recommendation for each individual temperature. The apparatus 100 may provide customized clothing recommendation information to the user despite the large daily temperature difference and unpredictable temperature change. While the conventional clothing recommendation technology uses a plurality of average pieces of information, the apparatus 100 may provide user-customized clothing recommendation information in consideration of different degrees of cold/heat perceived by individuals.

The apparatus 100 may provide individual customized clothing recommendation information rather than a plurality of averages. The device 100 may guide real-time temperature change at regular intervals. It provides information on the degree of cold/heat for specific clothes according to real-time temperature changes, and can provide clothing recommendation information using clothes currently owned by the user. In addition, the device 100 acquires (acquires) clothing information from other users who feel the same degree of cold/heat as themselves through the community, and generates clothing recommendation information using the obtained clothing information of other users. can provide The device 100 may acquire information without directly accessing the app through a widget function.

The device 100 may input information about owned clothing through camera recognition, and, for example, if it is impossible to use a camera, may input it by directly typing.

As an example, the device 100 may first provide average clothing recommendation information, and when information on owned clothing owned by the user is acquired and stored, personalized clothing recommendation based on pre-stored information on owned clothing Information can be generated and provided.

For example, the device 100 provides customized clothing recommendation information to the user by continuously and continuously asking questions about cold/heat (ie, an input request message for input of sensible temperature information) before sufficient data is accumulated. can improve the accuracy of

In addition, the device 100 may give an anonymous number (identification number) to prevent fighting with an anonymous person who is cherished in a community made only by anonymity. In addition, the device 100 may provide a shopping mall link so that when necessary clothes are available, they can be purchased more quickly.

Conventionally, in relation to clothing recommendation technology, there are many unknowns and related posts on clothing suitable for each temperature that are uploaded to the Internet, and there are many people looking for clothing suitable for the temperature in YouTube videos and the like. However, in the case of such clothing recommendation techniques, since the recommended clothing is often not owned by the person, it is difficult to actually use the recommended clothing recommendation information.

Accordingly, the device 100 intends to provide clothing recommendation information by utilizing the clothing (owned clothing) owned by the individual. The apparatus 100 may provide personalized clothing recommendation information focusing on an individual rather than a plurality of averages. The device 100 may provide a function to communicate with users similar to the user, and through this, it is possible to acquire (acquire) subjective information other than objective information, and comprehensively consider them to provide more customized clothing recommendation information. . Also, the apparatus 100 may receive clothing recommendation information anytime, anywhere in a situation (environment) in which GPS and data reception are possible, for example.

Conventional clothing recommendation technologies recommend based on the manufacturer's own standards or average clothing (clothing recommendation information) according to temperature. On the other hand, since the device 100 provides attire recommendation information (clothing recommendation information) based on individual app users, not based on the manufacturer or average, users' usage rate can be increased.

The device 100 may acquire information about owned clothing only by camera recognition or a photo. Also, the device 100 may provide, for example, common clothing recommendation information that can be worn by both men and women in addition to men/women. The device 100 may provide a widget function that can be viewed on the desktop of the user terminal 1 . The apparatus 100 may provide information on how the user feels at night with respect to the selected clothing by providing a temperature difference between night and day. In addition, the device 100 may be linked with a shopping mall, and may provide detailed clothing recommendation and clothing information. Also, the device 100 may set a specific situation in relation to feeling the temperature (feeling temperature information), and may provide a pop-up reminder function, a voice function, and the like.

The apparatus 100 may generate clothing recommendation information based on the current weather. The apparatus 100 may provide detailed information (eg, material, thickness, etc. of clothing) on the clothing recommendation information. The device 100 may generate clothing recommendation information based on the weather for each time period, or may generate clothing recommendation information based on the weekly weather and clothes selected by the user.

The apparatus 100 may provide average clothing recommendation information (clothing recommendation information) in consideration of the temperature. Also, the apparatus 100 may generate and provide clothing recommendation information for each time period, and may generate and provide clothing recommendation information based on daily (weekly information) as well.

When a user inputs an input request message for input of sensible temperature information, the apparatus 100 relates to information acquired in response to the time when the user input is made, weather information (weather forecast information), fine dust information, location Information, temperature difference information, etc. may be acquired, but the present invention is not limited thereto.

The device 100 may display, for example, an input request message for input of sensible temperature information on the screen in a pop-up format. Such an input request message may mean a message for requesting a user input in response to a query asking whether the user is currently cold or hot. Based on the user input to the input request message, the apparatus 100 may acquire information (sensible temperature information) regarding whether the user is currently cold or hot.

In addition, the device 100, through the simulation unit 270, when purchasing a fashion product (fashion product) sold on an online shopping mall, reduce the failure rate of the purchaser's purchase of the fashion product and provide a fashion product to be purchased effectively. have. In addition, the device 100 causes the simulation unit 270 to purchase and actually receive the product (product) on the online shopping mall, and the feeling/size of the product on the shopping mall is different from the feeling/size of the actual product Accordingly, it is possible to solve the problem of not having high satisfaction with product purchase and frequent exchanges/refunds.

Hereinafter, an operation flow of the present application will be briefly reviewed based on the details described above.

9 is an operation flowchart of a method for providing a social network service using augmented reality according to an embodiment of the present application.

The method of providing a social network service using augmented reality shown in FIG. 9 may be performed by the apparatus 100 described above. Accordingly, even if omitted below, the description of the device 100 may be equally applied to the description of the method of providing a social network service using augmented reality.

Referring to FIG. 9 , in step S11 , the avatar manager may generate a user avatar corresponding to the user and store the generated user avatar in the first storage unit.

Next, in step S12, when it is detected that the shooting mode is in the shooting mode, the screen controller may overlap and display the user avatar generated in step S11 on the real image displayed on the screen of the user terminal corresponding to the shooting target area.

Next, in step S13 , when a user input is made to the shooting button in the shooting mode, the image management unit may acquire an augmented reality image in which a real image and a user avatar are combined and store the acquired augmented reality image in the second storage unit.

Next, in step S14, when a sharing request signal for sharing the augmented reality image obtained in step S13 with other users is input from the user, the SNS management unit may upload the augmented reality image to the SNS relational network for sharing.

In the above description, steps S11 to S14 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present application. In addition, some steps may be omitted if necessary, and the order between steps may be changed.

The social network service providing method using augmented reality according to an embodiment of the present application may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

In addition, the above-described method for providing a social network service using augmented reality may be implemented in the form of a computer program or application executed by a computer stored in a recording medium.

The above description of the present application is for illustration, and those of ordinary skill in the art to which the present application pertains will understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present application. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present application is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present application.

1000: Social network service providing system using augmented reality
100: device for providing social network service using augmented reality
210: avatar management unit 220: screen control unit
230: video management unit 240: SNS management unit
250: first storage unit 260: second storage unit

Claims (12)

As a social network service (SNS, Social Network Services) providing device using augmented reality,
an avatar management unit generating a user avatar corresponding to the user and storing the generated user avatar in the first storage unit;
a screen control unit that overlaps and displays the user avatar on a real image displayed on the screen of the user terminal in response to a photographing target area when it is detected that the photographing mode is in;
an image management unit for obtaining an augmented reality image in which the real image and the user avatar are combined and storing the augmented reality image in a second storage unit when a user input is made to the shooting button in the shooting mode; and
When a share request signal for sharing the augmented reality image with other users is input from the user, an SNS management unit for uploading the augmented reality image to an SNS relational network for sharing;
A social network service providing device using augmented reality comprising a. According to claim 1,
the user is a first user, the user avatar of the first user is a first user avatar, the augmented reality image is a first augmented reality image,
The screen control unit,
If it is detected that the user terminal of the second user is in a photographing mode for photographing the same photographing target area as the photographing target area, and the first user avatar exists as a pre-stored user avatar in relation to the photographing target area, the An apparatus for providing a social network service using augmented reality to overlap and display a second user avatar and the first user avatar on a real image displayed on a screen of a user terminal of a second user. 3. The method of claim 2,
The video management unit,
Obtaining and storing a second augmented reality image as an augmented reality image in which the real image, the second user avatar, and the first user avatar are combined,
When a non-combination condition is previously set in relation to the first user avatar, combining for obtaining the second augmented reality image is performed in consideration of the uncombinable condition, providing a social network service using augmented reality Device. According to claim 1,
The avatar management unit,
generating the user avatar in consideration of body characteristic information input from the user;
The user avatar is a three-dimensional avatar generated in consideration of at least one of a user's body type, face shape, facial expression, and skin tone received as the body characteristic information, the apparatus for providing a social network service using augmented reality. 5. The method of claim 4,
The avatar management unit,
and an information acquisition unit configured to acquire at least one of the upload information analysis result calculated by analyzing the user's biometric information, weather forecast information, and upload information uploaded by the user when it is detected that the shooting mode is in the shooting mode,
The apparatus for providing a social network service using augmented reality, which updates status information of the user avatar based on the at least one obtained information, and generates and stores the updated user avatar. According to claim 1,
The screen control unit,
When the real image is displayed on the screen of the user terminal for a preset time, at least one preset recommended position is displayed on the real image, and any one preset recommended position from among the at least one preset recommended position Displaying the user avatar on the social network service providing device using augmented reality. 7. The method of claim 6,
The at least one previously selected recommended location is,
A position selected in consideration of object information identified through image analysis of the real image displayed on the screen and/or a display position of a user avatar calculated based on analysis of a plurality of augmented reality images uploaded to the SNS relational network A device for providing a social network service using augmented reality, which is a location that is selected in consideration of related statistical location information. According to claim 1,
The screen control unit,
When the real image is displayed on the screen of the user terminal for a preset time, by analyzing the uploaded information uploaded to the SNS relational network in relation to the real image in relation to the real image, a recommended visit to recommend a visit to a place located in the vicinity of the real image to determine whether location-related information exists,
If it exists as a result of the determination, a social network service providing apparatus using augmented reality to generate a manager avatar including information related to the recommended visit place and display the generated manager avatar on the real image. 9. The method of claim 8,
The screen control unit,
When a user input for the manager avatar is made, the information related to the recommended visit place is displayed in a pop-up form on one area of the screen of the user terminal,
When a plurality of place items are included in the recommended visit place-related information, the display order of the plurality of place items in the recommended visit place-related information displayed in the pop-up form is determined according to the user action information for the plurality of place items. A social network service providing device using augmented reality, which is updated in consideration of the level. 10. The method of claim 9,
The level of the user motion information is divided into a plurality of levels including a first level, a second level, and a third level,
The user action information corresponding to the first level is information about a user action of performing a selection input operation of selecting at least one place item from among the plurality of place items from a user terminal,
The user action information corresponding to the second level includes a route guidance request input for requesting a route guidance to a place corresponding to the one place item in relation to any one place item among the selected at least one place item Information about the user's action to perform the action,
The user action information corresponding to the third level is a place image acquisition operation of acquiring an image related to a place corresponding to the one place item based on the route guidance information generated in response to the route guidance request input operation A device for providing a social network service using augmented reality, which is information about a user's action to perform. As a social network service providing system using augmented reality,
A social network service providing apparatus using the augmented reality of claim 1; and
a user terminal interlocking with the apparatus for providing a social network service using the augmented reality to receive a control signal for controlling the operation of the apparatus for providing a social network service using the augmented reality as input from the user;
A system for providing social network services using augmented reality, including a. As a method of providing a social network service using augmented reality by the apparatus for providing a social network service using the augmented reality of claim 1,
(a) generating a user avatar corresponding to the user and storing it in a first storage unit;
(b) overlapping and displaying the user avatar on a real image displayed on a screen of a user terminal in response to a shooting target area when it is detected that the shooting mode is in the shooting mode;
(c) when a user input is made to the shooting button in the shooting mode, acquiring an augmented reality image in which the real image and the user avatar are combined and storing the augmented reality image in a second storage unit; and
(d) when a share request signal for sharing the augmented reality image with other users is input from the user, uploading the augmented reality image to an SNS relational network for sharing;
A method of providing a social network service using augmented reality comprising a.

Images